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Preface 


In 2006 we were working together at a well-established institution with 40 
years’ experience in conducting social and political surveys. Within our 
professional circle, the development of regular surveys of the general 
Spanish population posed few major challenges. At least one was con- 
ducted every month. However, an emerging development in Spanish soci- 
ety gradually began to demand more of our attention. This development 
created new challenges in survey design and, indeed, seemed to have the 
potential to change our professional lives. 

Field reports from interviewers increasingly mentioned that dwellings 
where they tried to do interviews were occupied by non-Spanish people. 
When analysing the results it became evident that the most common sur- 
veys, covering only the Spanish population, increasingly reflected a less- 
than-complete picture of society. In fact, they were missing an important 
new part of it. The public institutions for which we developed surveys 
were aware of the problem too. They began to ask us to broaden the target 
population to include foreign and immigrant populations and, in some 
cases, even to focus primarily on them. However, they did not realise the 
greater technical difficulties and much higher cost involved in doing this! 

The need to address these new issues convinced us to organise an inter- 
national workshop on the methodological challenges involved in surveys 
of immigrants and minorities. That workshop took place in Madrid in 
October 2008, with participants from nine European countries. The 
European Science Foundation sponsored the event and also provided fund- 
ing for preparation of this manuscript. The participants in the workshop 
presented analyses of various methodological issues (sampling, fieldwork, 
etc.) faced in surveys of immigrants and foreigners (or of general popula- 
tion surveys that include them). This book contains papers based on those 
presented at the workshop. A few of the Madrid papers did not evolve into 
chapters, but we want to thank their authors, Michael Blohm, Giancarlo 
Blangiardo, Dirk Jacobs, Vincent Tiberj, Orkeny Artal and Henk 
Stronkhorst (as the ESF’s representative) for their contributions to the dis- 
cussion of the issues covered here. We also include here several new con- 
tributions from European contexts (Denmark and Switzerland) and North 
America (United States). 

We thank the three IMISCOE anonymous reviewers who made excellent 
suggestions for improving the manuscript. 
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Lise Togeby was invited to the workshop, but had to cancel due to 
health problems. Sadly, she died shortly after. We therefore dedicate this 
book to Lise Togeby’s memory. 


Joan Font and Mónica Méndez 


1 Introduction: The methodological challenges 
of surveying populations of immigrant origin' 


Joan Font and Mónica Méndez? 


1.1 The purpose of this book 
The growing importance of international migration 


Migration flows around the world have increased rapidly in recent decades. 
The immigrant population in OECD countries has more than tripled since 
the 1960s. According to the United Nations (2009), some 3 per cent of the 
world’s people lived in a country other than the one they were born in in 
2010. 

Geographical mobility is an old phenomenon, but international migration 
has grown in volume and significance since 1945, particularly since the 
mid-1980s. As Castles and Miller (2009: 10-12) claim, one of the most dis- 
tinctive traits of the migration movements in recent decades has been their 
global scope. An increasing number of countries has been involved in mi- 
gratory movements, and at the same time the division between migrant- 
sending and migrant-receiving states is blurring. Compared to previous 
waves of migration, the current movements of populations across countries 
are more diverse in terms of migrants’ economic, social, cultural and politi- 
cal backgrounds, producing more differentiation of migration, and amplify- 
ing its potential social, cultural and political impact in receiving countries. 

Immigration has become a salient issue on the political agenda in many 
countries. This has led to an increasing demand for data, not only regard- 
ing flows of populations between countries, but also related to the charac- 
teristics and living conditions of migrants within countries (as well as their 
integration). This need to monitor population settlements is not new. As 
Reeger and Sievers (2009: 297) point out, the desire to have control over 
the population residing in a given territory and even the word ‘statistics’ 
are very much linked to the development of the nation-state in the nine- 
teenth century, though some of the tools used by statisticians, such as cen- 
suses, were already in use in ancient civilisations. However, censuses can- 
not satisfy the current demand for information, and surveys have become 
the most widely used data-collection tool. This book deals with the techni- 
cal and methodological challenges that surveying immigrant populations 
entails and how to confront them. Before addressing these challenges, the 
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next section looks in more detail at the demand for data with which these 
surveys are trying to comply. 


The greater demand for data on immigrants 


In European countries, the realisation that new waves of migrants who had 
arrived in the post-World War II period intended to remain permanently in 
the host countries increased the need to know more about these popula- 
tions. As Schmitter (1980) points out, it was soon clear that these migrant 
workers would become part of the host countries and societies. Thus, what 
had started as a temporary system of recruiting workers to satisfy demand 
for low-skilled labour was turning into permanent settlement. Even in the 
1970s, after the economic crisis became apparent, the number of immi- 
grants did not decline but increased, due, for example, to the process of 
family reunification. 

There has been a shift both in the political and in the academic realm 
from focusing mainly on migration flows to paying more attention to the 
consequences of the permanent presence of immigrant populations in host 
countries. In the political sphere this has meant a greater emphasis on poli- 
cies of integration (accommodation or assimilation, depending on the cases 
concerned). The transition from interest in studying flows to knowing more 
about the conditions for integration in different realms of the host society 
has increased the need for reliable data about the trajectories and character- 
istics of immigrants; the ways they live and think, their plans for the future 
and so on. All of these data, most of which is collected through surveys, 
are crucial to devising and evaluating public policies for fostering integra- 
tion and adaptation. To be sure, empirical research about immigration also 
uses other data-collection tools apart from surveys, but in this book we fo- 
cus only on the latter. 

The greater diversity of recent migration movements has enhanced the 
need for data as well. Though information on immigrants may be needed 
regardless of this diversity, the need may be more pressing because of the 
greater differences between the new population and the “autochthonous” 
one. Additionally, there is greater diversity among the newly arrived 
groups. Public policy and public services aimed at these populations need 
to be based on accurate information about them in order to plan and 
achieve successful outcomes. 

Diversity is not only greater as far as the background of immigrants is 
concerned, but also regarding the reasons behind migration. People migrate 
for different reasons and under different conditions: as manual workers, 
highly skilled workers, entrepreneurs, refugees or to reunite with relatives 
who had previously migrated. While the potential impact of migration is 
linked to its magnitude, it is amplified by the great diversity of migrants, 
which can bring about changes in the demographic, economic and social 
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structures of host countries and even question national identity through cul- 
tural diversity (Castles & Miller 2009: 15). Survey data from both autoch- 
thonous and migrant populations is needed to assess the extent to which 
these changes are actually taking place. 

This interest in gathering information is not restricted to newly arrived 
immigrants. Both in the traditional immigration countries such as the 
United States (Portes & Rumbaut 2001, 2005) and in European countries 
there is a need for reliable data regarding the descendants of immigrants 
(second and even third generations), as pointed out by several EU reports 
(Ramb 2007). Though in a different situation from their parents, second 
(and further) generations of immigrants potentially face similar challenges 
and difficulties in terms of social, economic and political integration 
(Bonifazi, Okólski, Schoorl & Simon 2008). 

Stimuli to obtain information about migration dynamics and migrants” 
characteristics come from international bodies as well. The United Nations 
and organisations such as the OECD and the World Bank have pointed out 
the need for better and more harmonised data on migrants, and they have 
engaged in a range of activities to achieve this goal. The European Union, 
particularly its statistical office Eurostat, has been moving in this direction 
as well. An example of such an initiative is the Task Force on Improving 
Migration and Migrant Data Using Surveys and Other Data, also referred 
to as the ‘Suitland Working Group”. 


The Suitland Working Group stemmed from a collaborative effort of the United Nations 
Economic Commission for Europe (UNECE), the World Bank, the US Census Bureau 
and the Conference of European Statisticians. Part of a work plan to improve migration 
statistics, its primary objective is to enhance the use of household surveys to measure 
levels and outcomes of migration. 


In March 2009, the US Census Bureau, along with Eurostat, UNECE, the United 
Nations Population Division, and the UK Office of National Statistics, sponsored a con- 
ference on using household surveys to measure migration and the size, distribution, 
and characteristics of migrant populations. This conference was held at US Census 
Bureau headquarters in Suitland, Maryland. 


Since its first meeting the group has had several meetings and developed several pa- 
pers to facilitate international collaboration, formulating a research agenda focused on 
methodological issues and creating reports accessible to a wide audience and especially 
addressed to statistical agencies. Their 2010 meeting showed an enhanced interest in 
cooperation and discussed several projects developed in this framework. 


www.unece.org/stats/groups/suitland/suitland.html 


Within the European Union, regulation (EC) 862/2007 on community 
statistics on migration and international protection in 2007 was a major 
step forward in reaching common definitions and standards regarding 


14 JOAN FONT AND MÓNICA MÉNDEZ 


migration statistics. Later, the Declaration of Zaragoza (2010) stressed the 
need to have common indicators to shed light on the different aspects of 
the living conditions of migrants and to monitor the integration policies 
carried out by member states. There are two major survey data sources for 
European migration statistics: 1) the European Labour Force Survey (LFS) 
conducted in the 27 EU member states and three candidate countries as 
well as three states of the European Free Trade Association (EFTA) (all ex- 
cept Liechtenstein) and 2) the EU Statistics on Income and Living 
Conditions (EU-SILC) conducted in the EU-27 (Kraler & Reichel 2010: 
28). Beyond those efforts to homogenise statistical data, the European 
Union has actively facilitated development of comparative research in the 
field, funding major projects like Promoting Comparative Quantitative 
Research in the Field of Migration and Integration in Europe 
(PROMINSTAT). 


PROMINSTAT is an EU-funded project to promote comparative quantitative research 
about migration and integration. It provides useful information about censuses, sur- 
veys, counts and registers in a fully searchable database that covers the 27 EU member 
states plus Norway and Switzerland. The database includes hundreds of microdata 
sources on numerous topics, from population to discrimination, employment and 
health care. It is primarily addressed to researchers, helping them to locate sources of 
data on these topics, providing information on data limitations and suggesting poten- 
tial avenues for comparative research using the data. 


The project has also produced a series of country reports and thematic studies (e.g. on 
migration flows and on citizenship). These reports review the available comparative da- 
ta for the different areas. 


www.prominstat.eu 


As pointed out by Kraler and Reichel in the PROMINSTAT final report 
(2010: 33-34), data-collection policies and historical development have in- 
fluenced countries’ choices among registers, counts and surveys as tools 
for collecting information on migration issues. For example, the Nordic 
countries make extensive use of registers, whereas countries like France, 
the Netherlands, Spain and Germany rely to a greater extent on surveys. 
Poland, Ireland, the United Kingdom and Estonia tend to rely more on stat- 
istical counts than on surveys. 

Preference for one data-collection method over others is also influenced 
by the topic being examined. For instance, information about the social 
and political attitudes of migrants cannot really be obtained through regis- 
ters; surveys are clearly the best candidate means of getting these types of 
data. In other topic areas, such as the participation of migrants in the la- 
bour market, while there might be good register data in many countries, a 
survey might be a useful complementary data source when looking for 
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potential explanations of different patterns of participation in the labour 
market or when new variables that are not present in registers need to be 
introduced as central explanatory factors. 

Surveys tend to be a versatile means of data collection in terms of both 
the array of topics that can be covered and the adaptability of questionnaire 
and sample designs to address a particular research question in a particular 
setting. Whereas censuses and registers tend to be data-collection opera- 
tions organised regularly by national statistical institutes, sometimes specif- 
ic information on a topic can be achieved only by designing an ad hoc 
survey. 

To sum up, the demand for more data (or for any data in some countries 
where no alternatives exist) has given surveys a major role among data-col- 
lection tools in obtaining the information needed. This pressing demand 
occurs at a time when survey methodology is well developed. But practi- 
tioners have nonetheless found that methodology has to be adapted and 
specific strategies developed to meet the particular requirements entailed in 
surveying immigrants (and ethnic minorities). 


The challenges involved in surveying immigrants 


Researchers confront serious challenges in producing reliable data about 
minority groups, and particularly migrant, foreign born or ethnic minor- 
ities. To start with, many surveys of the general population or of particular 
groups have a sampling frame with which to build the sample, but this is 
often not the case in surveys addressed to immigrants. As this book will 
show, only in a limited number of cases will we have access to a reliable 
list, containing country of origin, nationality or ethnic identity as well as 
the other necessary information to contact the individuals in our universe. 
In some countries, this disaggregated information does not exist. French 
legislation, for example, does not allow incorporation of the ethnicity of a 
given individual in any statistical file. In other countries, such as Italy, this 
information may exist, but is rarely available to researchers. In a third 
group of cases, this information exists and can be made available, but con- 
tains substantial errors, due to the greater residential mobility of the group 
and to the illegal residential status of some members of the group. 

The same logic of much greater difficulties, compared to performing 
these tasks in surveys addressed to the general population, applies in other 
phases of the survey process. Today, all social and political surveys face 
the problem of increasing non-response rates associated with difficulties in 
contacting the respondents and in obtaining their cooperation (Groves 
2006, Stoop 2005). The situation when interviewing immigrants may not 
be radically different in its basic components, but there are additional fac- 
tors that might make things even harder: if many citizens are difficult to 
reach because of greater residential mobility and complicated daily 
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agendas, these two circumstances will only be aggravated when dealing 
with immigrants, whose residential patterns are likely to be radically more 
unstable and whose lifestyles will produce even shorter pertods of avail- 
ability for Interviewing at home. If the number of interviewees who refuse 
to respond because they do not like surveys, do not trust them, or simply 
do not have time is high among almost any social group, it is likely to be 
higher in groups that tend to have experienced racist attitudes or may have 
a limited knowledge of the official language spoken in the country of 
residence. 

Subjects that make some respondents uneasy with survey questions exist 
for almost any citizen. Questions about income, intimate behaviours, vot- 
ing and political opinions may evoke defensive attitudes, partial non- 
response or even a refusal to participate in surveys among many segments 
of the population. However, the list of potentially difficult themes is longer 
when we are talking about surveys addressed to immigrants. In some cases, 
this is simply due to cultural differences, whereby a topic considered un- 
controversial in a given culture may be considered a sensitive one in a dif- 
ferent setting. In other cases, the greater difficulty of surveys addressed to 
immigrants may be due to the questions they include, which may refer to 
sensitive aspects of the circumstances of the interviewee (legal status, so- 
cial security affiliation and so on) which they might be reluctant to reveal 
to an unknown person that has just rung the doorbell. 

In short, to produce good survey data is always difficult. But these diffi- 
culties increase enormously when dealing with population segments that 
do not always appear on official statistics, that are more difficult to reach, 
that have good reasons to distrust surveys and that are not fluent in the 
host country’s main language. 

As the next section will show, much effort has been devoted to produc- 
ing data on numerous aspects related to immigration and the social and po- 
litical challenges that ethnically plural societies face. Explaining the usual 
survey process, its contents, problems and strategies to produce quality re- 
sults has also been the goal of dozens of extraordinarily good books. 
However, we still lack a systematic account of the existing challenges and 
strategies to produce reliable survey data on immigrants and ethnic minor- 
ities. This book looks at these different strategies and the specific chal- 
lenges involved in surveying migrants. It does not look at other relevant is- 
sues regarding the overall data quality, comparability of data or issues re- 
garding data analysis. 

The need to produce data, given the importance of the social problems 
to be addressed, has resulted in most efforts being focused on creating and 
interpreting results, at the expense of discussing how data have been pro- 
duced and their quality. When we have dozens of weekly polls in a pre- 
election context, a debate may arise about whether some of the data are 
more reliable than others and why. However, in a context of scarce 


INTRODUCTION 17 


information (e.g. about the social and political integration of immigrants), 
any new data is likely to be received as crucial input and limited attention 
given to how the dataset has been generated.” 

In most cases, previous literature using surveys of immigrants does not 
provide in-depth accounts of the particular strategies followed to carry out 
the surveys and their effects.* As far as we know, there is only one book 
fully devoted to explaining the methodological details of a survey ad- 
dressed to immigrants (DaVanzo, Hower, Burciaga & Vernez 1995). This 
book is a rich and suggestive account of a survey of immigrants, yet it 
deals with a single pilot experience developed more than ten years ago in a 
city in the United States. Lessons derived from this experience may be use- 
ful, but they hardly cover the diversity of situations that immigrant surveys 
may face concerning the local or national realities they aim to describe, the 
organisations and the resources available to develop the survey and the le- 
gal and statistical frameworks, to mention just a few examples. Two more 
recent books in the IMISCOE series make interesting contributions to the 
field, respectively, presenting migration-related data by country (Fassmann, 
Reeger & Sievers 2009) and offering a methodological discussion of vari- 
ous aspects of migration research (Bonifazi et al. 2008).° 

Our goal is to take a decisive step towards filling this gap in the litera- 
ture. This book provides methodological analyses, results and discussions 
dealing with more than a dozen different surveys with extremely different 
scopes, subjects and budgets. The surveys are from different countries, but 
all have one thing in common: they deal with immigrants or ethnic minor- 
ities (see next section). Each of these surveys is primarily focused on a giv- 
en subject (health, social and economic situation, political participation, 
etc.), but the difference between this book and others is that here the reader 
will not find much information about those topics. Contrary to most of the 
previous literature, we will not deal with the substantive results of these 
surveys, but with the methodologies followed, the difficulties faced and the 
strategies undertaken to (try to) solve them. Our aim is to describe, to 
document and to discuss how immigrant survey data are produced, what 
special difficulties are faced and what the results are of adopting particular 
research strategies. Into how many languages is it worth translating a ques- 
tionnaire? Is interviewer ethnic matching the best strategy in all cases? Are 
purely random sampling strategies the best possible ones in all circumstan- 
ces? The presentation and discussion of the diverse set of immigrant sur- 
veys covered in this book will help produce answers to these and other 
questions that anyone having to deal with immigrant surveys has to face. 

Surveys dealing with immigrants and with ethnic minorities are the fo- 
cus of this book. Many of the difficulties one confronts when carrying out 
this type of survey are shared by other surveys, particularly those that are 
mainly addressed to any kind of minority or difficult-to-reach group. What 
are the implications of choosing a particular target population for sampling 
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design? Sampling the general population or sampling a specific group for 
which we have a sampling frame (e.g. doctors or judges) is a relatively 
simple task. However, if we are in France, for example, and we want to do 
a survey of Algerians or if we want to do a survey of “Greens”, we sud- 
denly face certain difficulties. In both cases, we know that the target popu- 
lation represents a sizeable part of the population and we may even know 
that they live more in certain areas than in others. But, in both cases, we 
first need an operational definition of who the target population is, who is 
an ‘Algerian’ (someone born in Algeria? whose parents were born in 
Algeria? who feels Algerian?) and who is a ‘Green’ (someone who voted 
for them in the last presidential election? someone who is a party member? 
who identifies as a Green?). Secondly, we need a strategy to build a sample 
frame, to stratify it and to choose the final interviewees in a rigorous, but 
feasible way. 

A similar thing can happen with languages. The United Kingdom, Spain 
and Lithuania (to mention just a few) are Western countries with important 
linguistic minorities. In these countries surveys are often carried out only 
in the most important official language of the country, the language which 
most citizens are able to understand and to speak. If we are only interested 
in aggregated national figures, it may not be worth translating question- 
naires into ‘minority’ languages, as these populations will very often speak 
the main national language: the likelihood of finding a Scottish citizen 
who has trouble with English or a Catalan citizen who has difficulties with 
answering a questionnaire in Spanish is relatively small. But, for example, 
given their limited number among the general population, Chinese immi- 
grants who do not understand the official language are likely to make up 
only a very small proportion of any individual survey, but they are likely 
to constitute a large proportion of the total Chinese community living in 
the country. The dilemma whether it is worth translating a questionnaire in- 
to one of these languages (e.g. Scottish or Chinese), how to do it, using 
what procedures and with which interviewers is similar in both cases. 
Whether the two groups are migrants, or cultural minorities, does not have 
significant implications in this regard. 

As a consequence, when dealing with surveys addressed to immigrants 
one faces specific dilemmas common also to other types of surveys, espe- 
cially those designed to study any kind of minority population. The main 
characteristic of immigrant surveys is that most of the problems and diffi- 
culties associated in general with surveys are more likely to occur and in a 
more severe way. Thus, looking again at the example of a survey whose 
target is the Scottish population (a minority in the context of the whole of 
the United Kingdom), the fact that this population is geographically con- 
centrated facilitates the fieldwork being carried out and organised in a cul- 
turally sensitive manner. In contrast, with immigrant populations territorial 
dispersion and social exclusion (and some associated effects like high 
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residential mobility and lack of trust) are likely to be higher than among 
the autochthonous Scottish population, increasing the difficulties usually 
associated with different phases of the survey process. 

This introduction has two additional sections. The next one looks more 
closely at the objective of the book: surveys of migrants and minorities 
and general surveys that aim to include these populations. The final section 
deals with the logic, structure and content of the book. 


1.2 Surveys of immigrants and immigrants in general 
population surveys: A diverse landscape 


Defining the universe: Immigrants, foreigners and ethnic minorities 


Surveys that we are interested in might be addressed to authorised or to un- 
authorised immigrants, to foreigners or to ethnic minorities, just to mention 
a few potential target populations. The definition of who constitutes the tar- 
get group of a survey has to be made by the team in charge of its design 
and organisation, depending on the research goals. That said, the decision 
may also be linked to the data available. This, in turn, may be associated 
with the historical characteristics of immigration in a given country. For 
example, in the United Kingdom the term most widely used when design- 
ing surveys is ‘ethnic minorities’. Ethnicity is thus the main concept driv- 
ing both the research goals and the definition of target groups. This may 
have to do with the fact that the United Kingdom is a longstanding immi- 
gration country, so looking at recent immigrants would not allow research- 
ers to focus on the sociological issues regarding integration that they are 
interested in. As chapter 2 in this volume recalls, in contrast to other coun- 
tries, the UK statistics on ethnic minorities are gathered on the basis of 
self-identification among pre-established categories in the population cen- 
sus that 1s carried out once every ten years (Jacobs et al. 2009: 80-82). 

France is similarly a “traditional” country of immigration, but ethnicity is 
hardly ever the defining trait of a target group;” rather, migrant origin is 
most widely used so as to include not only recent immigrants, but also 
their descendants (Cusset 2006). This has been primarily the case since the 
1990s. Earlier, most statistics gathered and empirical research carried out 
was on the basis of nationality (the distinction being made between those 
that had nationality by birth or by acquisition) (Tribalat 1989). 

In the Netherlands, a new term, allochtoon, has even been coined to re- 
fer to persons who come from other countries and have settled permanently 
in the country, as well as to their descendants. The definition of the term 
has undergone several changes in official statistics. It is currently used to 
refer to people born abroad of whom at least one parent was also born 
abroad and also to people who were born in the Netherlands but had two 
parents born abroad (Jacobs et al. 2009: 79). 
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In the ‘new’ countries of immigration, there is still no great difference 
between addressing surveys to immigrants or to foreigners, though in some 
cases the rapid process of naturalisation (and the different citizenship re- 
gimes depending on the origin of migrants) makes it necessary to be pre- 
cise regarding the goals of the research and the way to define the target 
population in order to attain them. 

In short, the target group for this type of survey may vary in name (and 
this usually reflects different research goals and even “immigration” tradi- 
tions), but the methodological challenges that these surveys face are similar 
and for the purpose of this book can be treated interchangeably. 


A dual theme: Surveys of immigrants, immigrants in surveys 


This book deals with two quite different kinds of realities. Whether we are, 
for example, conducting a survey of the Senegalese community that resides 
in Paris or developing a survey that has as its target population all residents 
in Paris (and consequently, also including the Senegalese living in Paris) 
the challenge of surveying immigrants arises. In both cases we need to de- 
cide whether our response categories will accurately reflect the meanings 
and concepts used by the respondents and, in both cases, we will have to 
decide whether it is better to use interviewers of Senegalese origin to in- 
crease the participation of Senegalese in the survey. However, many of the 
most important problems will be different: in surveys addressed exclu- 
sively to Senegalese residents, for example, one of the main challenges will 
be how to sample in order to achieve a good representation of the group, 
whereas the second type of survey (that of Paris residents, including 
Senegalese) does not need to represent the group fully but only that they 
appear in the global picture of Paris society. Thus, the problem in the latter 
example is whether it is worth paying any special attention to covering the 
Senegalese population, or whether they constitute so small a part of the 
Paris population that no special considerations should be adopted and their 
presence will thus depend solely on random selection. 

In the next two sections we want to present these two different realities. 
First, we will deal with surveys addressed basically to immigrants and mi- 
norities, for example, surveys created to measure and capture their realities. 
Second, we will address the presence of immigrants in surveys that are de- 
signed to cover the general population. 


Surveys of immigrants: A general mapping 


Surveys of immigrant communities have probably grown as fast as the 
size and the social, economic and political relevance of the groups they 
aim to represent. For example, in the consumer sector there are products 
for which immigrants constitute important markets and, as a result, 
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quantitative and qualitative studies have been carried out to identify mar- 
keting strategies. 

Initial research efforts to capture the social reality of immigrants were 
often based on ad hoc local studies, which sampled the immigrant popula- 
tion through non-random selection and made intense use of qualitative ap- 
proaches. However, the last decade of the twentieth century and the first 
decade of the twenty-first century saw a boom in the study of immigrants, 
with a second generation of approaches being developed, including several 
quantitative surveys using random sampling procedures aiming to ad- 
equately represent immigrant communities and capture specific facets of 
their social realities. 

Reasons for conducting surveys that are exclusively addressed to immi- 
grants may be to answer specific questions that relate primarily to these 
populations (e.g. on migration trajectories) or because too few migrants are 
included in general population surveys to allow a proper analysis (Kraler 
& Reichel 2010). 

Most of the surveys are cross-section done in a particular country (re- 
gion or city) with the aim of gaining knowledge of the characteristics of 
migrants or a subgroup of them. There are many examples of this type of 
survey in different countries, such as the Latino National Political Survey 
in the United States. 


The Latino National Political Survey was developed for the first time in 1989-1990, with 
a second edition in 2006 (Latino National Survey). The main goal of the survey was to 
study the political attitudes and policy preferences of the self-identified Latino popula- 
tion with a sufficiently large sample size (8,634 interviews) to allow for a state-level 
treatment of the data. 


This major study now co-exists with several others, with important differences in their 
scope and objectives. First, the Latino Ethnic Attitude Survey has a shared interest in 
the (subjectively defined) Latino population, but is much more focused in its goals and 
sampling objectives: its explicit goal is to examine the content and meaning of Latino 
identity. 


www.icpsr.umich.edu/icpsrweb/ICPSR/studies/6841 detail 


Similar surveys exist in other countries. For example, the British 
National Survey of Ethnic Minorities has already reached its fourth edition. 
This was carried out between 1993 and 1994 with a sample of 5,196 adults 
(16 years of age and older) who had Caribbean, Indian, Pakistani, 
Bangladeshi or Chinese family origins and also with a white comparison 
sample group made up of 2,867 individuals. 

Special attention is warranted for longitudinal surveys of new (recent) 
immigrants. Such research is considered to be particularly suited to moni- 
toring the process of integration of immigrants in the host societies (Jacobs 
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2010). The pioneer study of this type is the Longitudinal Survey of 
Immigrants to Australia (LSIA) launched in 1994 and later replicated in 
2000 and 2005. A similar survey design was used in the Canadian 
Longitudinal Survey of Immigrants (2001), the US New Immigrant Survey 
(2003) and the Longitudinal Immigration Survey: New Zealand (LisNZ), 
which started in 2004. In the European context a recent survey of similar 
characteristics was started in France in 2010.* 


ELIPA (Longitudinal Survey of the Integration of First-Time Arrivals)? is a panel survey 
project developed by the French government to monitor a group of 6,000 people who 
had received a first residence permit and wished to settle permanently in France. These 
people were first interviewed in 2010. The project involves re-interviewing the same in- 
dividuals again in 2011 and in 2013. 


The goal of the survey is to gain reliable data on immigrant integration in France. The 
questionnaire includes questions about their migration, life and working conditions, ed- 
ucational and professional trajectories and other personal and social issues. Interviews 
are performed in thirteen languages by a private company (TNS-Sofres). Partial funding 
for the project comes from the European Union. 


www.immigration.gouv.fr 


These longitudinal surveys of immigrants all share similar goals: to ob- 
tain data about the settlement experiences of recently arrived immigrants 
and to understand how they adapt to life in each of these countries (Table 
1.1). Having a longitudinal design to collect data about the same individu- 
als over time, these studies provide a better picture of the settlement proc- 
ess than would be captured with a survey done at one point in time. They 
all have shortcomings, too. For example, since the definition of the target 
population in most of them refers to immigrants who have obtained a visa 
or some kind of residence permit to stay in the country, they exclude the ir- 
regular and undocumented immigrants, who precisely for this reason might 
experience a much tougher process of settlement. Other longitudinal panel 
studies, such as the Children of Immigrants Longitudinal Study, have fo- 
cused not on new immigrants but on the descendants of immigrants 
(Portes, Fernandez-Kelly & Haller 2009). 

Finding out about causes and consequences of international migration 
requires a special type of survey. In order to understand the reasons for mi- 
grating and to analyse the factors that facilitate this decision it is important 
to investigate both migrants (in destination countries) and non-migrants (in 
origin countries), thereby carrying out what Rallu (2008) refers to as ‘both- 
way’ surveys (also called bi-national/multi-site surveys) in origin and desti- 
nation countries. 

Among the surveys carried out with this type of design we can mention 
the 1987 survey done within the Mexican Migration Project developed in 
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the United States and in Mexico (Massey 1987), the 1993 REMUAO sur- 
vey!9 that covered eight sending and receiving countries in Africa and the sur- 
vey Push and Pull Factors in International Migration, carried out between 
1994 and 1999 under the leadership of the Netherlands Interdisciplinary 
Demographic Institute (NIDD, which covered five sending countries in 
Africa and three host countries in Europe (Rallu 2008: 275). Some of these 
surveys are done as ‘parallel’ studies in origin and destination countries. 
Others attempt to provide a link between the samples at the origin and des- 
tination. This means that, for example, a Mexican migrant will be inter- 
viewed in the United States and some of his/her relatives (and friends) will 
also be interviewed in Mexico. These types of surveys provide interesting 
information, but they entail methodological difficulties, especially in regard 
to the sampling process, fieldwork organisation and the challenge of 
achieving representativeness of migrants in the destination country 
(Beauchemin & González-Ferrer 2011). 


MAFE is a research project launched in 2008 that focuses on migration between sub- 
Saharan Africa and Europe. It covers the flows between Senegal and France, Spain and 
Italy, between the Democratic Republic of Congo and Belgium and the United 
Kingdom, and between Ghana and the United Kingdom and the Netherlands. Issues in- 
vestigated include the changing migration patterns from Africa to Europe, the reasons 
for emigration and return, the impact of international migration on work and the lives 
of migrants and non-migrants and their families. The sampling design and fieldwork or- 
ganisation have faced several challenges, due to the different sample frames (or lack 
thereof) in the various countries considered. 


A biographic questionnaire, virtually identical in all countries, is administered to 5,700 
individuals. In sending countries both non-migrants and returned migrants are inter- 
viewed, while in receiving countries the focus is on both documented and undocu- 
mented migrants. Data are collected from households (in the sending countries) and 
from individuals (both in origin and destination countries), using separate question- 
naires. 


www.mafeproject.com 


The comparative analysis of migration processes and integration also 
entails carrying out co-ordinated surveys in different countries. This is the 
most recent trend in  European-funded projects such as 
LOCALMULTIDEM'' (presented in chapters 6 and 7 of this book) and 
TIES. The project already mentioned carried out by NIDI in the 1990s and 
the more recent MAFE combine a survey done both in origin and destina- 
tion countries with a cross-country comparative design. In this type of en- 
deavour the methodological challenges of surveys addressed to ethnic mi- 
norities/immigrant communities are added to the usual concerns found in 
designing surveys to be administered in different countries: finding equiva- 
lence in the target groups to be interviewed, achieving a common 
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questionnaire, translation issues, comparability of method of data collection 
and so on (Harkness, Braun, Edwards, Johnson, Lyberg, Mohler, Pennel & 
Smith 2010). 


TIES (The Integration of the European Second Generation) is a comparative research proj- 
ect started in 2005, whose aim is to study the integration patterns of the descendants 
of immigrants from Turkey, ex-Yugoslavia and Morocco in fifteen cities across eight 
European countries (Germany, France, Austria, Switzerland, Belgium, Sweden, Spain 
and the Netherlands). 


The data used to achieve this goal is gathered through a survey of the same groups in 
various European countries, using the same questionnaire and definition criteria of 
what constitutes the second generation. The groups surveyed were the target group, de- 
fined as native-born children of Turkish, Moroccan or former Yugoslavian immigrants in 
the age group from eighteen to thirty-five years old, while the comparison group con- 
sisted of native-born children of native-born parents in the same age group and neigh- 
bourhood as the former. As in other comparative projects, sample frames differ among 
the different countries. 


The questionnaire includes modules on educational background, labour history, family 
relations, housing and neighbourhood, social relations, gender roles and child care, 
identity language and transnationalism, religion and other socio-demographic details of 
respondents. 


www.tiesproject.eu 


This limited set of examples is enough to show the ambitious efforts that 
have been developed to carry out surveys with immigrants as their target 
population. There are some important differences among them. The defini- 
tion of the target population, the sampling sizes and strategies, the cross- 
sectional or panel structure, the modes of administration and the main sub- 
jects of interest differ, making them important complementary studies that 
explore diverse questions and help us to understand different aspects of im- 
migrants’ realities. The chapters in this book provide other examples which 
show how diverse research and policy-related interests stimulate the use of 
contrasting methodological strategies. 


Immigrants in general population surveys: A general mapping 


Immigrant populations not only create their own realities (that must be 
analysed through the specific surveys just described), but they also trans- 
form the overall societal picture. Going back to the example mentioned 
at the beginning of this section, the municipality of Paris may have an 
interest in better knowing its Senegalese community to incorporate their 
specific needs into local policies; but it also needs information about 
how a growing immigrant population is changing the social composition, 
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policy needs and attitudinal preferences of the general population in 
Paris. 

We will review the incorporation of minorities in three different types of 
surveys: household panels, cross-sectional socio-political surveys and the 
major international comparative surveys, like the World Values Survey or 
the European Social Survey. 

Large surveys used to elaborate social and economic statistical data, 
such as the EU Labour Force Surveys, the International Social Survey 
Programme usually incorporate migrants/non-nationals and have large 
samples, so it is possible to analyse them as a subgroup. For this reason, 
they have been considered by the European Union to be an important 
source of information to obtain indicators about the integration of 
immigrants. 


A special module of questions to analyse the participation of migrants and their de- 
scendants in the labour market was carried out in the 2008 EU Labour Force Surveys 
(EU-LFS) in all member states (Ramb 2007). The evaluation report of the survey 
showed that the response rate among immigrants is lower than that among the autoch- 
thonous population, and non-nationals tend to be under-represented with respect to of- 
ficial figures from censuses and population registers, but having large samples the 
quantitative implications of this under representation are less severe than in other 
cases (Eurostat 2010). 


The example of the EU-LFS 2008 ad hoc module demonstrates that adaptation of gen- 
eral population surveys to include migrants/non-nationals is not merely a matter of 
sampling design: the module carried out in 2008 dealt not only with an increase in 
sample sizes, but also with incorporation of a set of questions that facilitated identifica- 
tion of different types of migrants and descendants of migrants. 


http://circa.europa.eu/irc/dsis/employment/info/data/eu_lfs/ 


The main panel household surveys usually incorporate immigrants in 
their target population, but they must also take into account demographic 
changes over time (such as immigration flows and changes in migrant 
group composition). All have adopted mechanisms to ensure an accurate 
representation of immigrants. Some, like the United States’ Panel Study of 
Income Dynamics (PSID), take as a point of departure the wider aim of 
having a better representation of the overall country’s population, beyond 
sampling immigrants with the goal of doing subgroup analysis focused on 
them. The PSID, for example, incorporated a refresher sample in 1990 to 
include individuals who had arrived in the country from Mexico, Puerto 
Rico and Cuba after 1968 (when the survey first started). While this new 
sample allowed for a better representation of some immigrant groups, it left 
out other important ones, such as Asians, so it was dropped in 1997-1998. 
In its place came another representative sample of 500 immigrant families 
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that had arrived or were born in the United States after 1968 (PSID 2011). 
Some changes were also made to the German Socio-Economic Panel 
(SOEP) to keep its sample aligned with immigration flows (Haisken- 
DeNew, John & Frick 2005). While the immigrants who had arrived in 
(West) Germany in the 1960s were already part of the panel’s initial sam- 
ple design,'? in 1994 and 1995 two new samples of immigrants were 
added. 

In other panel household surveys, such as the United Kingdom’s 
Understanding Society, there is a boost sample in order to produce a larger 
sample size for analysis of five relevant ethnic groups (Indian, Pakistani, 
Bangladeshi, Caribbean and African) (see Berthoud, Fumagalli, Lynn & 
Platt (2009) and chapter 2 in this volume). In this case, the aim of having 
a boost sample was not just to gain a better representation of the overall 
population of the United Kingdom, but also to be able to do specific analy- 
ses of these five ethnic minorities. This could be viewed as an immigrant 
longitudinal survey nested within a general population survey, to allow de- 
tailed analyses of these populations in a general and comparative 
framework. 

Whatever their main goal, long-established surveys have needed to adapt 
to the new reality by changing sample designs and data-collection strat- 
egies. New projects, like the Dutch Longitudinal Internet Studies for the 
Social Sciences (LISS) panel, have adopted from the start a methodological 
strategy that recognises this multi-ethnic societal composition. 


The Dutch LISS panel consists of 5,000 households, comprising 8,000 individuals. The 
panel is based on a probability sample of households drawn from the population regis- 
ter by Statistics Netherlands. To ensure coverage of people who do not use the 
Internet, the individuals selected to participate in the panel receive PCs and broadband 
Internet access if they do not have a computer and/or Internet. 


LISS included migrants in their original sample, but due to lower response rate they 
were under-represented among final respondents (Knoef & De Vos 2009: 11-12). In 
2010 a new immigrant panel was initiated in addition to the LISS panel. A control 
group of native Dutch was included in this new panel, which comprises around 1,600 
households (2,400 individuals) of which 1,100 households (1,700 individuals) are of 
non-Dutch origin. 


www.lissdata.nl/lissdata 


Attitudinal and public opinion surveys seem to lag behind those that 
seek to collect data about socio-economic indicators, such as household so- 
cio-economic panels. This might have to do with the fact that, although 
not exclusively, public opinion surveys have often been carried out some- 
what linked to elections or attempts to assess the impact of governmental 
policies in electoral competition. The most established general population 
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attitude surveys, such as the General Social Survey (GSS) in the United 
States and the British Social Attitudes (BSA) survey in Great Britain, in- 
clude immigrants in their target population, but they do not usually have 
boost samples of immigrants that allow for any detailed analysis of these 
groups. In most countries, surveys that aim to forecast votes rarely include 
immigrants, since the right to vote is usually restricted to citizens; when 
non-nationals have the right to vote, it is generally only in local (or possi- 
bly, regional) elections. 

Even if slower in time, it is important to mention the symbolic impor- 
tance of the incorporation of immigrants in attitudinal surveys: being a part 
of a group that is allowed to express opinions on public issues is a signifi- 
cant step toward acquiring some of the rights associated with citizenship. 
Through attitudinal surveys, the opinions and values of immigrants are in- 
corporated into public voices (like those of any other citizen) and may 
have an effect on public policies or at least in the public debate. In con- 
trast, the incorporation of immigrants has been much faster in surveys 
geared at obtaining social and economic indicators or those that aim to 
measure the needs and demands of public-service users, such as health or 
education (where immigrants are becoming a crucial public). 

However, surveys that aim to achieve full and serious coverage of immi- 
grant and minority populations should consider other issues apart from the 
definition of the target population, such as the use of languages other than 
the main official national language. Whereas most surveys of immigrants 
use other languages, most general population surveys continue to use only 
the official national language. For example, whereas surveys like the BSA 
and ALLBUS” only include residents with a sufficient knowledge of the 
native language (Blohm 2008), since 2006 the US General Social Survey 
(GSS) has become an exception to this pattern by using a Spanish version 
of its questionnaire in addition to the English one (see chapter 10 in this 
volume). 


The German General Social Survey (ALLBUS) is a consolidated general survey on social 
and political attitudes that has developed more than fifteen editions. When the survey 
was widened to the population of unified Germany in the early 1990s, the universe was 
also changed from voters to residents, thus also including foreign-born adult residents. 


Questionnaires and interviews are performed only in German. Turks, the largest immi- 
grant group in Germany, are estimated to be less than 2 per cent of the population, 
which would mean organising a huge effort for about 60 interviews (the second lan- 
guage group being Italians, in a much smaller proportion). Since their interest is the 
overall national picture, differences due to missing segments of the immigrant popula- 
tion in the final sample would only appear in very specific questions. 


www.gesis.org/en/allbus 
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This means that, in practical terms, even if the theoretical universe of 
these surveys is the adult resident population, in reality the absence of spe- 
cial provisions regarding language in some of them restricts the universe to 
the adult resident population able to speak the national language well 
enough. This is unlikely to create problems regarding minorities from for- 
mer colonies or from countries that use the receiving country’s majority 
language (Algerians in France, Ecuadorians in Spain and so on), but it will 
mean the exclusion of significant parts of the newly arrived populations 
coming from places with very different languages (such as the Chinese in 
Western Europe). For example, returning to municipalities and examining 
the case of Barcelona, all surveys of the general population carried out by 
the city administration since 2003 claim to be covering the resident popula- 
tion. However, a detailed look at the methodology followed reveals that, 
for example, a telephone survey that excludes mobile phones and uses no 
languages other than Catalan and Spanish produces an average of 6-7 per 
cent of foreign-born people in the sample; however, some 20 per cent of 
the total adult population in Barcelona is estimated to be foreign born.'* 

International comparative surveys have also incorporated immigrants, al- 
beit to various extents depending on the particular survey being considered. 
For example, looking at three of the most well-known comparative attitudi- 
nal surveys, we observe quite different patterns. The oldest one, the World 
Values Survey, has no clear homogeneous rules: universe and sampling de- 
tails change from one country to another and from one wave to another. 
Most countries seem to follow the pattern just described: they aim to cover 
all the resident population, but do not always use different languages in 
questionnaires or interviews. For example, Table 1.2 summarises the use of 
foreign languages in the last wave of the survey. An enormous number of 
countries (81 per cent of the 56 countries participating in the survey) use 
only a single language. Many of these countries are largely monolingual 
(e.g. Argentina), but in others the use of a single language excludes de fac- 
to a sizeable proportion of the population, whether this is due to immigra- 
tion (United States or European countries) or to the historical linguistic 


Table 1.2 Number of interviewing languages used in the last wave of the World 
Values Survey (selected examples) 


Number of Percentage Examples 
countries 
One language 41 82 USA, Netherlands, Mali, China 
Two languages 6 11 Georgia, Canada 
Three languages 4 7 Malaysia, Taiwan 
Four languages 2 4 Andorra, Ghana 
Five or more languages 3 5 India, South Africa 


Source: Authors’ own elaboration. Data from www.worldvaluessurvey.org 
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diversity of the country (e.g. French being the only language used in the 
interviews carried out in Mali or Morocco). 

A similar pattern is found in other important comparative surveys, such as 
the International Social Survey Programme (ISSP), which also aims to cover 
all the residents in each participant country, but has no homogeneous rule on 
the use of additional languages. The European Social Survey (ESS) also cov- 
ers all resident population in the countries where it is carried out. It has de- 
veloped a general rule regarding languages: when there is another language 
spoken as a first language by at least than 5 per cent of the population of a 
participating country, the questionnaire should be translated into that second 
language; as a result, four languages are used in Luxembourg, three in 
Switzerland and Russian is also used (outside Russia itself) as an interview- 
ing language in Ukraine, Israel and Estonia, to mention just a few cases. 

As this set of examples shows, the incorporation of all the resident pop- 
ulation has been increasingly implemented into not only national, but also 
comparative (general population) surveys. The necessary adjustments to 
ensure that this incorporation goes beyond merely changing the definition 
of the target population, have been more limited because of the increased 
costs and organisational complexity involved. However, carrying out the 
methodological changes needed to adapt to multicultural societies is be- 
coming an additional indicator of the rigour and quality of any representa- 
tive survey, and the most serious efforts in this direction developed in the 
ESS are an indication of its commitment to rigorous methodological 
standards. 


1.3 The subject, logic and plan of this book 
The subject: The methodological challenges of surveying immigrants 


In the previous section we have already pointed out some of the special 
difficulties faced in surveys of immigrants. The next pages present the 
most important of these difficulties, addressed in two parts: (1) difficulties 
associated with sampling and defining the universe and (11) difficulties as- 
sociated with the data-gathering process (features of fieldwork like lan- 
guages, interviewers and response errors). In both cases, we will consider 
how these difficulties appear and are coped with in the context of surveys 
primarily addressed to immigrants, and in the case of general population 
surveys that include immigrant populations. 

Surveys have been considered a special kind of conversation. However, 
they are a very unnatural conversation, where one of the participants essen- 
tially questions and the other answers (Converse $ Schuman 1974). In 
fact, the understanding of the survey situation can be reduced to these two 
participants: the interviewer (and the organisation that stands behind her) 
and the interviewee, who is providing the answers. We will first focus our 
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attention on the respondent and then on the interviewing organisations and 
the resources they provide. 

Starting with the interviewee, the first crucial decision to make is what 
population we are interested in — that is, the target population of the sur- 
vey. An initial issue in this discussion, is the definition of a migrant or eth- 
nic minority, as discussed in the previous section. However, there are other 
crucial decisions to be made, beyond the choice of focusing on the particu- 
larities of immigrant groups or on those who consider themselves as (sub- 
jectively) belonging to an ethnic group different from the majority. For ex- 
ample, are there relevant subpopulations among this target group about 
whom we want to know more? The answer to this question is never an 
easy one: there may be immigrants from more than forty countries living 
in the Netherlands. Ideally, we might like to know about each of them be- 
cause, for example, we suspect the circumstances of Somalians to be quite 
different from those of Ethiopians. However, we are not likely to have the 
resources to achieve representative samples (i.e. about 400 interviews) of 
each of these forty nationalities (totalling 16,000 effective interviews). 
Almost all of the very different surveys covered in this book have faced 
this dilemma. Are there specific groups with a large enough population so 
that their presence should be guaranteed through the sampling design? 
What specific groups are these? Can countries, languages or cultures be 
grouped somehow, so that, even if representation cannot be achieved for 
each group individually (e.g. for Ecuadorian and Peruvian immigrants in- 
dependently), it can at least be done by aggregation according to geograph- 
ical or cultural affinities among them (e.g. Andean). The chapters in this 
book will explain the dilemmas that different surveys faced in addressing 
this issue and the decisions they made. Chapter 2 will discuss in greater 
detail the problems involved and the trade-offs of a wide array of sampling 
strategies. 

The decision about whom to survey is not only related to the definition 
of the groups we want to know more about. Very often, survey results by 
themselves are of limited interest and their richness only appears when 
they are placed in comparative perspective: comparing two countries, the 
same population across time or comparing results from two ethnic groups 
in a given city helps us to assess the real value of the data and to interpret 
the results. Surveys addressed to immigrants often reveal two different 
realities: that found among ethnic/immigrant groups and that found among 
the autochthonous population (Jacobs 2010). Chapters 6 and 8 are based 
on research designs that purposely incorporated a control group of people 
of Danish/Swedish origin to make this important comparison possible. 

Once the decision has been made about which subgroups to include, sur- 
veyors have to answer the same questions as in any other regular survey: 
is there an adequate sampling frame? How far from perfect is it, and which 
populations are going to be excluded or under-represented if we use it? 
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The situation regarding surveys addressed to ethnic minorities/immigrants 
is not inherently different from other surveys, but in most circumstances it 
is likely to be more complex. Sometimes, adequate sampling frames do not 
exist for any kind of survey, but such a possibility is more likely if we are 
dealing with minorities, with a particularly mobile population group, with 
situations including illegal residence status, or other factors associated with 
(severe) social exclusion. Since all of these factors are very likely to be 
linked to recent immigration status in many Western countries, more often 
than not, surveys of immigrants will lack an adequate sampling frame (or 
at least there will be reasonable doubt about its adequacy). 

One problem that may emerge in many countries is that the information 
available is not what is needed. In France, for example, no information re- 
lated to ethnicity is allowed in any official records and as a result, finding 
an appropriate sampling frame for certain types of surveys is difficult. In 
other countries there may be data, but it may not provide the information 
we are interested in (e.g. country of birth may be available but without in- 
formation on parents’ country of birth, thereby impeding the identification 
of second-generation immigrants). In addition, sometimes data exists but is 
only available for official purposes and therefore not provided to individual 
researchers. 

This book provides examples of different approaches regarding sampling 
frames. Chapter 4 presents a case in which the research team found that no 
appropriate sample frame existed and had to create one; whereas chapter 7 
presents a case where an adequate sampling frame did exist, but research- 
ers nonetheless had to adapt it to the resources available to carry out the 
fieldwork. In contrast to these cases, the research team in charge of similar 
surveys in Sweden and Denmark (see chapters 6 and 8) were largely satis- 
fied with the sampling frames used. Problems faced in several of the sur- 
veys mentioned in this book are the higher residential mobility of migrant 
populations!” and immigrants living at ‘non-official’ addresses (e.g. people 
sleeping in cars) (chapter 4), to mention only two. 

Sometimes no satisfactory sampling frame can be found and the survey 
project has to be abandoned. In other cases where no sampling frame is 
available, the need for the data is so acute that resources can be provided 
to construct one. Several considerations arise at this point. If there are good 
estimates of the distribution of the target population in small areas, a sam- 
ple frame can be built if there are sufficient resources to map the areas 
(chapter 2). The lack of this information will make the task difficult and al- 
most force a clearly non-probabilistic approach to the problem. Chapter 4 
is an interesting example of the first kind of situation, where good data 
about the social and spatial distribution of the immigrant population al- 
lowed a probabilistic selection of sampling areas and non-probabilistic 
methods had to be used for the final selection of respondents. In fact, the 
approach used for this final selection was quite similar to the Centre 
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Sampling method that has been widely used in the Italian context (Mecatti 
2004, Blangiardo 2008). A completely different strategy that has also often 
been used is the selection of foreign names from telephone directories: for 
example, the European Internal Migrations Social Survey (EIMMS) identi- 
fied 250 interviewees in each of the five largest European countries in this 
way, to study people who had migrated between 1973 and 2003 
(Santacreu, Rother & Braun 2006). Obviously, this strategy will be prob- 
lematic if foreign names cannot be distinguished easily enough and will as 
a likely result miss people without phones and foreign wives of nationals, 
to mention just two categories. 

In short, sampling may be very difficult in surveys addressed to immi- 
grant populations. Of all the problems that may be faced, this book will ba- 
sically address three: 1) how to define the groups and subgroups in which 
we are interested, 2) how to know if we have a good sampling frame and 
the limitations associated with using an existing one, and 3) what methods, 
probabilistic or not, can be used to make a more or less rigorous sample if 
this sample frame does not exist or does not provide enough information. 
The book will cover quite different scenarios that will hopefully provide 
valuable insights to be more fully discussed in the conclusion. 

The second subset of questions to be considered here are those con- 
cerned with survey organisation and the resources used to achieve a suc- 
cessful interview. We will again address three main aspects: 1) question- 
naire development (including language and translation issues), 2) selection 
of interviewers, and 3) the study and management of non-response. 

The questionnaire is the special ‘conversation’ code that guides the inter- 
view situation. In the usual survey methodology book it would be one of 
the central aspects to discuss. In this book, questionnaires do not play such 
a crucial role.'* This does not mean that there are no relevant issues re- 
garding questionnaire development, such as making sure that the concepts 
used have a similar meaning across the different cultural backgrounds of 
ethnic/immigrant minorities covered. Thus, we will discuss how the ques- 
tionnaire development stages might be different from other surveys and 
what can be learned from a central step, the pilot study. Chapters 3 and 6 
illustrate the substantial benefits that can be extracted from a good pilot 
study to improve the questionnaire. However, as has been mentioned in the 
previous section of this chapter, perhaps the most central point related to 
questionnaires in surveys of immigrants are languages and translation is- 
sues:'’ the use of fewer languages will mean a de facto exclusion of parts 
of the population. 

In most cases, it will be impossible to include every language that is 
spoken as a first language by any member of the population, but costs, in- 
terests and language abilities will have to be considered to make a bal- 
anced decision. Chapter 10 about the General Social Survey demonstrates 
the consequences of including the second most important language in the 
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United States. This chapter also shows that even when considerable resour- 
ces are available, translation is not an easy task: even the strictest double 
checks may continue to produce a lack of strict comparability among lan- 
guages like English and Spanish or among the specific meanings that a 
given adjective has for people of Chilean and Guatemalan origin, even if 
they speak the same language. 

The translation of questionnaires into one or more other languages is the 
central strategy to include a wide range of the population that does not 
speak the main language of the country fluently. Only through a serious 
and controlled translation can we guarantee the highest level of meaning 
equivalence between concepts and ideas (Harkness 2003). However, trans- 
lation may be a necessary but not sufficient condition to achieve greater 
representativeness, that is, to not exclude de facto parts of the sample. First 
of all, the availability of questionnaires in a second language means that in- 
terviewers must be able to use that language (except in the case of self-ad- 
ministered surveys). Second, even translating questionnaires, the fear re- 
mains that socially excluded communities may be reluctant to answer inter- 
viewers who do not belong to their group, or at least they may provide 
them less reliable answers (Schaeffer 1980; Reese, Danielson, Shoemaker, 
Chand & Hsu 1986). The need to obtain high cooperation rates has intro- 
duced the practice of what has been called ‘ethnic matching’, that is, trying 
to achieve the maximum correspondence between the nationalities of inter- 
viewers and interviewees. Several chapters in this book address that issue, 
but it is the central concern of chapter 5, which will introduce important 
critical arguments about this practice. 

In most societies, the debate about ethnic matching has wider implica- 
tions, as there are not enough professional interviewers belonging to each 
of the ethnic minorities. As a result, in practice, ethnic matching leads to a 
wider debate concerning the use of professional interviewers: is it better to 
use professionally trained interviewers or people who may have less expe- 
rience but can generate greater trust and produce more empathy from inter- 
viewees? The researchers in this book have again used different solutions 
to this dilemma and reach (partially) contradictory conclusions about the 
ideal solutions. 

The main reason for using interviewers who may facilitate communica- 
tion with the respondents is their suspected impact on response rates, the 
final topic of this book. The response rate of any survey is considered one 
of the most important indicators of its quality and reliability. Too low a re- 
sponse rate means that an important part of the original sample has not 
been reached and there may be good reasons to fear that those surveyed 
may differ from those not included (Groves 2006). Non-response among 
immigrants was rarely studied until recently (Deding, Fridberg & Jakobsen 
2008; Feskens, Hox, Lensvelt-Mulders & Schmeets 2006). Non-response 
is in fact the combination of two different situations: people who cannot be 
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reached (non-contact) and people who refuse to answer (non-cooperation). 
Throughout this book and elsewhere there is general agreement that contact 
rates are lower among immigrant groups, but competing empirical claims 
are made regarding cooperation, both in previous literature as well as in 
this volume. 


The logic and structure of the book: Challenges, cases and lessons 


How is the information in this book organised? Our starting point has been 
the identification and discussion of the main challenges in survey research 
among ethnic and immigrant minority populations. What do these chal- 
lenges mean? What are their consequences and the possible strategies to 
face them? We believe that presenting these challenges and strategies in a 
vacuum would not be very useful. Each survey and each research project 
is a multifaceted reality encompassing a national reality, a given set of hu- 
man and economic resources and a specific set of research objectives; to 
present all the pieces of this comprehensive situation is, in our view, the 
best way to give the reader a general sense of the conditions under which a 
given research strategy was adopted. 

As a result, the structure of the book follows a dual logic, combining a 
problem-oriented and a case-oriented approach. Thus, the first thread 
around which the book is organised relates to the specific problems that 
come up in surveys addressed to ethnic or immigrant minorities and the 
possible strategies to address them. Following this logic the book is div- 
ided into three main parts. The first part primarily covers sampling issues. 
The main focus of the second part is fieldwork organisation and develop- 
ment. Part three of the book centres on the presence of immigrant/ethnic 
minorities in general population surveys. 

Returning to the idea that each survey is a reality of its own, each chap- 
ter in the book is also case-oriented, presenting an individual research proj- 
ect or survey that has immigrants or ethnic minorities as its main target 
population or a general population survey that is also addressed to immi- 
grants. The discussion focuses primarily on problems related to certain 
methodological challenges and strategies depending on which part of the 
book the chapter appears in. Organising the information in such a manner 
allows us to deal more in-depth with each specific challenge (sampling, 
languages and ethnic matching of interviewers, to mention just a few) 
without losing sight of the overall reality of each of the surveys. This helps 
us to understand the context in which the decisions regarding research de- 
sign were made. 

Thus, the goal of the book is not to present a set of particular survey sto- 
ries. Rather, the book aims to be a useful tool for any person interested in re- 
search about immigration who wants to evaluate the risks and pitfalls of a 
given research strategy, or who is considering how to organise her own 
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research or how to judge the validity of the immigrant survey data she has. 
With this goal in mind, each of the chapters and particularly the book’s con- 
clusion discusses the lessons to be learnt from survey experiences. These 
lessons will surely not be simplistic (e.g. just because a particular sampling 
strategy was a success once does not mean that it should be used in every 
circumstance), but a great deal can be learnt from each of the small prob- 
lems, successes and limits of this group of quite diverse survey experiences. 

The first part of the book includes three chapters that concentrate on 
sampling issues. Chapter 2 presents different alternatives to elaborate prob- 
ability (and non-probability) sampling of minority ethnic groups. It covers 
the main issues in designing these types of samples: problems in classify- 
ing the population on the basis of ethnicity, deciding which ethnic groups 
to survey or to boost and the way to carry out the samples. It emphasises 
the difficulties in achieving nationally representative surveys of all ethnic 
groups, particularly when there are financial and practical constraints and 
how to achieve compromises between ideal and feasible goals. Chapter 3 
discusses the experience of the 2007 Spanish National Immigrants Survey 
(END. Š The sample design, the changes introduced as a result of a diffi- 
cult pilot experience and some of the most important features of the field- 
work are presented. Thanks to a substantial organisational and financial op- 
eration and an adequate sampling frame, Spain has its first reliable data on 
the social profiles of its most important immigrant groups. If chapters 2 
and 3 represent different scenarios for surveys that had significant resour- 
ces for their completion, the final chapter of this first part is an example of 
how to organise an immigrant survey in a much more constrained scenario, 
when no adequate sampling frame is available and organisational con- 
straints (particularly time) are acute. Chapter 4 describes a case where it 
was necessary to build a much more original sampling strategy combining 
probabilistic and non-probabilistic features, which resulted (as the chapter 
comparisons with other official statistics show) in an appropriate picture of 
immigrants in Andalusia. 

The second part is made up of four chapters all dealing with surveys of 
immigrants, but concentrating on different kinds of fieldwork problems. 
Chapter 5 addresses one important aspect of fieldwork in any survey of mi- 
norities: ethnic matching. The Dutch survey discussed in that chapter used 
an experimental matching of interviewer’s strategy to learn whether higher 
cooperation rates and more reliable answers were obtained when interview- 
ees were interviewed by individuals from their country of origin. The re- 
sults cast doubt on the appropriateness of ethnic matching strategies for all 
circumstances, particularly when sensitive topics related to the native cul- 
ture are involved. Chapter 6 presents two related surveys of immigrants in 
Stockholm: one of immigrants and their descendants and another of immi- 
grant associations. The chapter reviews most of the problems that any sur- 
vey of immigrants will have to address, from interviewer recruitment to 
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questionnaire design and translation and the role played by the pilot study. 
Finally, the chapter presents its outcomes regarding response rates, which 
1s the main subject of the next two chapters. 

Chapter 7 presents another survey developed in the context of a research 
project that surveyed immigrants and immigrant associations in several 
European cities. The chapter focuses on two aspects of fieldwork (inter- 
vlewer training and languages); but its main subject is the analysis of re- 
sponse rates in the Madrid survey of five immigrant groups defined by 
place of birth. The chapter provides a critical analysis of the difficulties 
faced by most immigrant surveys developed by small organisations with 
limited resources available to carry out their survey. The last chapter of this 
section, chapter 8, based on a Danish survey, has a central focus on the 
question of why non-response occurs. How is it related to the personal 
characteristics of the respondents and of the interviewers? The chapter 
presents a set of hypotheses that are tested through a survey covering adult 
immigrants from three countries (Iran, Pakistan and Turkey) as well as na- 
tive-born Danes. The analysis of contacts, cooperation and response rates 
reveals the specific factors that most decrease the likelihood of individuals 
participating in surveys. 

The chapters in the third and final part of the book have a different pri- 
mary research objective: they are not focused on capturing the specificities 
of immigrant realities, but rather, on placing them in a broader picture of 
the overall society. The chapters in this section deal with general popula- 
tion surveys that include immigrant populations. Chapter 9, on the Spanish 
case, presents a broad picture, describing how immigrants have been grad- 
ually incorporated into the general population surveys of Spain’s main pub- 
lic survey institution, the Sociological Research Centre (Centro de 
Investigaciones Sociológicas, CIS). The chapter addresses the difficulties 
that arise in a research institution that must work efficiently and has no 
specialisation in issues regarding immigration, but must incorporate them 
into its sampling and fieldwork strategies. 

The chapter also presents the results of an experiment developed to test 
how many immigrants were being missed in CIS surveys and the impact 
of the use of Spanish as an interviewing language (even when the resident 
population was the target rather than individuals holding Spanish national- 
ity). This last issue is the main focus of chapter 10, which presents the ex- 
perience of the GSS in the United States when it incorporated the possibil- 
ity of conducting interviews in Spanish. The chapter explains the proce- 
dures used by the GSS to guarantee language comparability and evaluates 
the wider coverage of the new GSS and the implications of not using 
Spanish for the validity of results. 

Chapter 11 looks at the representation of foreign minorities among the 
respondents of three Swiss general surveys. Both cross-sectional as well as 
longitudinal surveys are used to carry out non-response and attrition 
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analysis in order to understand the reasons behind the under-representation 
of foreign minorities with respect to the Swiss population. Their analysis 
concentrates on the different patterns of under-representation of each of 
these foreign nationality groups and explores whether these patterns disap- 
pear once we control for factors like education and occupation. 

The concluding chapter goes back to the challenges presented earlier 
and analyses how they were addressed by the different surveys covered 
throughout the book. This chapter discusses the strategies employed and 
the importance of the contextual factors of each of the surveys (availability 
of sampling frame, budget or time constraints, survey main goals and 
others). The chapter sketches a number of useful tips to consider when 
planning a survey that covers immigrant population and discusses whether 
these recommendations can be generalised or are dependent on the context 
and ambitions of each specific survey. 


Notes 


1 We thank Amparo González-Ferrer for her useful comments on a previous version of this 
chapter. 

2 The views expressed in this chapter are personal and do not necessarily reflect those of 
the organisations to which the authors belong. 

3 To illustrate this point, even an interesting review of the state of the art of empirical re- 
search on immigration such as the one by Givens (2007) does not refer to the difficulties 
that obtaining good empirical data on the field entail and to the limitations of existing 
data. 

4 We are not claiming that previous literature on the subject does not explain the basic 
methodological steps followed in the research. We just argue that in most cases, methodo- 
logical concerns have not been the researchers’ main objective. As a result, these issues 
have not been dealt with in depth. 

5 The current volume complements the other two books. Fassman et al. (2009) cover sev- 
eral important European countries but miss the Spanish case, which is present in several 
chapters of this volume. The methodological chapters in parts 3 and 4 of Bonifazi et al. 
(2008) deal with those issues that are only marginally covered in this volume: the analysis 
of second-generation migrants and the use of coordinated surveys in the countries of ori- 
gin and destination. 

6 At least two important international conferences have been held on this subject. See, for 
example, the 2004 Symposium: Innovative Methods for Surveying Difficult-to-Reach 
Populations, organised by Statistics Canada (see www.statcan.gc.ca/bsolc/olc-cel/olc-cel? 
catno=11-522-XIE2004001&lang=eng) and the 2008 International Conference on Survey 
Methods in Multinational, Multiregional, and Multicultural Contexts (3MC) (see 
www.3mc2008.de). 

7 See Simon and Clément (2006) for an exception to this statement. 

8 In the UK some consideration was given to the possibility of undertaking a longitudinal 
survey of immigrants (Nazroo, Berthoud, Erens, Karlsen & Purdon 2005), but the idea to 
establish it as an autonomous project was disregarded and the project was integrated in 
the larger Understanding Society, a general population longitudinal panel household sur- 
vey (Berthoud, Fumagalli, Lynn & Platt 2009). See chapter 2 in this volume. 

9 Etude Longitudinale sur I’Integration des Primo-Arrivants (ELIPA). 
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10 Réseaux Migrations et Urbanisation en Afrique de l’Ouest (REMUAO). 

11 Multicultural Democracy and Immigrants’ Social Capital in Europe: Participation, 
Organisational Networks, and Public Policies at the Local Level. 

12 Sample A of the SOEP designed in 1984 covers persons in private households with a 
household head who does not belong to the main foreigner groups of guest workers (e.g. 
household heads who are Turkish, Greek, Yugoslavian, Spanish or Italian). These last 
groups have their own oversample (sample B using ‘SOEP’s terminology’), that allows 
for separate analyses of this population (Haisken-DeNew et al. 2005). 

13 Allgemeine Bevólkerungsumfrage der Sozialwissenschaften. 

14 Official data from Instituto Nacional de Estadística (INE, Spanish National Statistics 
Institute) January 2009. This strong discrepancy between the figures reveals a survey 
practice that incorporates about a third of the real adult immigrant population. 

15 For example, it led the ENI survey to change its sampling unit, from individuals to house- 
holds (see chapter 3). 

16 Another important issue that will not be covered in depth in this book is the mode of ad- 
ministration. The surveys examined in this book use various modes of administration, but 
additional research is needed to address this topic more thoroughly. 

17 The Eumagine Project, developed in several African and European countries with non-for- 
malised written languages, is an especially interesting example of how to deal with these 
challenges (Ersanilli, Carling & Haas 2011 and www.eumagine.org). See also Harkness et 
al. (2010). 

18 Encuesta Nacional de Inmigrantes. 
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SAMPLING ISSUES 


2 Designing high-quality surveys of ethnic 
minority groups in the United Kingdom 


Bob Erens 


21 Introduction 


Carrying out surveys among ethnic minority (EM) groups raises a number 
of problems above and beyond those to do with surveys of the general 
population. These difficulties arise because EM groups may appear with a 
low frequency in the population, may be geographically unclustered, and 
may be difficult to access. 

This chapter examines some of the key issues to do with designing rig- 
orous, high-quality surveys of ethnic minorities, whether as a ‘boost’ to in- 
crease their numbers in a general survey or as a targeted survey among par- 
ticular EM groups. By “high quality”, the survey needs not only to provide 
accurate data, but it also must relate to the needs of the users (e.g. it should 
be relevant, timely and accessible). While having indicators of survey qual- 
ity is Important — for example, to evaluate the usefulness of the data 
provided and to differentiate between ‘good’ and ‘bad’ data — this chapter 
is not concerned with issues of general survey design or quality (for a gen- 
eral discussion of survey methodology, see Groves, Fowler, Couper, 
Lepkowski, Singer & Tourangeau 2009; Fowler 2008; Lyberg, Biemer, De 
Leeuw, Dippo, Schwarz & Trewin 1997). 

The tension between cost and quality that is found in all surveys is even 
more acute when carrying out surveys of ethnic minorities, since these spe- 
cialist surveys tend to be more expensive in terms of cost per participant. 
Therefore, an EM survey is likely to cost more than a survey among the 
general population of a similar size and quality. This will need to be re- 
flected in the survey budget (unless reductions in sample size or quality 
are acceptable). 

The following sections in this chapter cover approaches to categorising 
EM groups and the classification commonly used in surveys in the United 
Kingdom; the practicalities influencing decisions about which EM groups 
to include in surveys; methods for sampling EM groups, including an exam- 
ple of how a probability sample can be designed cost-effectively using strat- 
ification and clustering; and data-collection issues including response rates, 
the translation process and ethnic matching of fieldworker and respondent. 
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22 Categorising the population of interest 


Before designing a survey sample, the key question that needs to be an- 
swered is who the survey population is. This refers not only to which EM 
groups to include, but also what is meant by an “EM group” in the first 
place, and how these EM groups can be clearly identified for the purposes 
of the survey (e.g. for sampling). 

A number of approaches have been taken to identify ‘racial’ or ‘ethnic’ 
groups, some of which have proven controversial. Over different time peri- 
ods and in different countries, “race” or “ethnicity” has been defined using 
one or more different characteristics, such as country of birth (own or pa- 
rents”), nationality, language, religion, skin colour, cultural traditions and 
ancestral origins. Bulmer (1999) has defined an “ethnic group” as follows: 


[a] collectivity within a larger population having real or putative 
common ancestry, memories of a shared past, and a cultural focus 
upon one or more symbolic elements which define the group's iden- 
tity, such as kinship, religion, language, shared territory, nationality 
or physical appearance. 


Moreover, ethnic identity should not be seen as static, but as socially 
contingent, flexible, and liable to change as social circumstances change. 
For the concept to be meaningful, it is essential that members of an EM 
group are conscious of being part of that group, and that they choose the 
characteristics they wish to use to define themselves (e.g. skin colour, an- 
cestry, language), although their choices may be influenced by the stereo- 
types imposed by others (Karlsen & Nazroo 2006). 

Given the complexity of defining ethnicity, there is no single or straight- 
forward way that EM groups can be easily identified, particularly when it 
comes to operationalising the categories for the purposes of a survey. 
When carrying out a survey, interviewers (and respondents) need to be able 
to clearly identify who is eligible for interview. Some characteristics may 
be easy to identify (such as country of birth), while others are subjective 
and more difficult to identify in a survey context (e.g. cultural traditions). 
No ethnic classification system is perfect, and none is universally accepted 
within the United Kingdom. 

The difficulty has been partially overcome by asking people to classify 
themselves as belonging to a particular EM group. The UK’s Office of 
National Statistics (ONS) designed an ethnic classification system for the 
1991 population census which was based on self-defined ethnicity: 


[Because] membership of an ethnic group is something that is sub- 
jectively meaningful to the person concerned [...] we are unable to 
base ethnic identification upon objective, quantifiable information 
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as we would, say, for age or gender. And this means that we should 
rather ask people which group they see themselves as belonging to. 
(ONS 2003: 9) 


ONS subsequently revised the self-identity question for the 2001 census, 
and this question, as described in Figure 2.1, is now widely used in nation- 
al government-funded surveys in the United Kingdom. 

For any self-completion survey, the question should be asked in the 
same format as that used on the 2001 census. For a face-to-face survey, 
these categories are usually included on a show card, with the question 
read out. For a telephone survey, two questions are asked: firstly, the re- 
spondent selects one of the main categories (e.g. Asian or Asian British) 
and secondly, the detailed groups within the selected main category are 
read out and the respondent selects the appropriate one (e.g. Indian, 
Pakistani, Bangladeshi, any other Asian background).' 

ONS has urged researchers in the United Kingdom to use these ethnic 
categories. While there are a number of criticisms of this classification, 


Figure 2.1 Self-identity question included in the 2001 UK census 


What is your ethnic group? Choose ONE section from A to E, then tick the appropriate box 
to indicate your ethnic group. 


A White 

British 

Irish 

Any other white background, please write in 
ixed 

White and Black Caribbean 

White and Black African 

White and Asian 


Any other mixed background, please write in 


M 


Asian or Asian British 

Indian 

Pakistani 

Bangladeshi 

Any other Asian background, please write in 
ack or Black British 

Caribbean 

African 


Any other black background, please write in 


a 


hinese or other ethnic group 
Chinese 


Any other, please write in 


Source: UK Office for National Statistics 
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such as including all “Black Africans” within a single category (Elam & 
Chinouya 2000), there are significant advantages for surveys to use the 
ONS categories, not least of which is to provide standardisation across 
surveys to enable them to be compared with each other. Other important 
benefits are that census data can be used to help design a cost-effective 
sample (as described below) and the representativeness of an achieved 
sample can be judged by comparing it with census results. While the 
ONS classification may not meet all needs, using this classification does 
not preclude further refinement of the categories — for example, looking 
at particular Black African groups or White EM groups — or the use of 
alternative classifications (e.g. based on language spoken or religion) 
within particular studies. 


23 Deciding which ethnic minority groups to include in a 
survey 


Identifying the EM groups to be included in a particular survey may of 
course directly arise from the aims of the study. For example, the objective 
may be to look at the characteristics of one (or more) specified EM groups 
within a particular locality. On the other hand, many large-scale national 
surveys wish to boost EM groups in a way that will obtain robust data that 
can be compared with the general population, at the same time as provid- 
ing reliable data about each of the EM groups included in the sample. In 
such cases, it may be decided either 1) to limit the EM groups sampled to 
the most populous groups in the country in order to provide robust data 
about each sampled EM group or 2) to include all EM groups, but to aim 
for sufficient numbers to provide reliable estimates only for the most popu- 
lous EM groups (while the remaining groups are put in a catch-all “other 
EM group” category for analysis purposes). Both approaches have been 
used in large surveys in the United Kingdom. 

Population data is likely to play a significant role in determining which 
EM groups can be practically and cost-effectively sampled in sufficient 
numbers to allow separate analysis. ONS publishes (experimental) yearly 
estimates of the population in England? by ethnic group. Table 2.1 shows 
the ethnic composition of the population of England in 2007.° 

As is evident from Table 2.1, in a large national survey in England of, 
say, 10,000 individuals, there would be only about 580 Asian respondents, 
of whom about 260 would be Indian, 180 Pakistani and 70 Bangladeshi. 
The number of Black, Mixed and Chinese respondents would all be con- 
siderably lower.* Since most analyses will wish to look at results by gen- 
der, age and other key variables, these numbers are clearly insufficient for 
separate analysis within individual EM groups. 
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Given these constraints, many national surveys in the United Kingdom 
limit their focus to the six most populous non-White EM groups, including 
Indian, Pakistani, Black African, Black Caribbean, Chinese and 
Bangladeshi groups. Examples of national surveys with boosts limited to 
the largest EM groups include the Fourth National Survey of Ethnic 
Minorities in Britain, carried out in 1993-1994 (Smith & Prior 1996); the 
1999 and 2004 Health Surveys for England (Erens, Primatesta & Prior 
2001a; Sproston & Mindell 2006); the 2000 National Survey of Sexual 
Attitudes and Lifestyles (Erens, McManus, Field & Korovessi 2001b); and 
the British Crime Survey up to 2001 (later years boosted all non-White EM 
groups) (Bolling, Grant & Sinclair 2006-2007). 

A recent example is the very large-scale survey called Understanding 
Society (formerly referred to as the UK Household Longitudinal Survey, or 
UKHLS) that was launched in January 2009 and aims to interview 


Table 2.1 Mid-2oo7 population estimates in England by ethnic group 


Ethnic group Mid-2oo7 Percentage 
population estimates’ 
White: 
British 42,736.0 83.6 
Irish 570.5 1.1 
Other White 1,776.3 3.5 
Mixed: 
White and Black Caribbean 282.9 0.6 
White and Black African 114.3 0.2 
White and Asian 260.9 0.5 
Other Mixed 212.0 0.4 


Asian or Asian British: 


Indian 1,316.0 2.6 
Pakistani 905.7 1.8 
Bangladeshi 353.9 0.7 
Other Asian 339.2 0.7 


Black or Black British: 


Black Caribbean 599.7 1.2 
Black African 730.6 1.4 
Other Black 117.6 0.2 


Chinese or other ethnic group: 


Chinese 400.3 0.8 
Other 376.1 0.7 
All England 51,092.0 100 


Source: ONS (2009) 
1 Figures in thousands 
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100,000 individuals from 40,000 households every year. Understanding 
Society includes an EM boost sample designed to achieve interviews with 
at least 1,000 individuals from the five main non-White EM groups (ex- 
cluding the Chinese), and as many members of other EM groups identified 
during the screening.” 


2.4 Sampling ethnic minority groups 


As illustrated above, a simple random sample drawn from the general pop- 
ulation will result in too few respondents from EM groups to permit analy- 
sis within these specific groups. So other means must be found for boost- 
ing EM respondents, while maintaining the representativeness of the over- 
all sample and of each EM group. 

Unlike some European countries, the United Kingdom does not have a 
comprehensive list that includes the ethnicity of all residents in the country. 
The Electoral Register (ER) was often used to draw samples until about 
twenty years ago, but today it is rarely used because of under-representa- 
tion of eligible voters and the opportunity for voters to opt-out of having 
their names included on the ER that is publicly available. EM groups are 
less likely than White British residents to be included on the ER, particu- 
larly as a proportion of EM residents are not UK nationals and thus not eli- 
gible to register. Thus, for sampling EM groups, the ER does not provide a 
good frame to start with. 

Moreover, since the ER does not include ethnicity, the only way EM res- 
idents could be sampled from the ER would be by surname. This applies 
to any list that might be contemplated as a sampling frame but which does 
not include ethnicity (such as patient lists and the telephone directory). 
While some researchers have suggested that sampling by surname is possi- 
ble for certain EM groups (e.g. for Chinese or South Asian surnames), this 
approach is far from infallible. Not only are some individuals with the 
sampled surnames not members of the EM group, but this approach ex- 
cludes everyone from the EM group whose surname is not on the list, in- 
cluding married women who may have changed their name. Moreover, this 
approach is not feasible for certain EM groups, such as Black Caribbean. 
Sampling via surnames is often a last resort when there are no other cost- 
effective solutions — for example, the Health Survey for England sampled 
Chinese respondents on the basis of surnames in the absence of a practical 
alternative. 

In the United Kingdom, the list most commonly used for sampling for 
high-quality surveys is the Postcode Address File (PAF), as it offers almost 
complete coverage of private residential households and is regularly up- 
dated. One problem with the PAF is that it cannot be used to sample insti- 
tutions (e.g. hostels and nursing homes), and thereby excludes a small 
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proportion of the population (although this could potentially be a large per- 
centage of the group of interest if, e.g. the group is older adults). 

But the main difficulty with the PAF is that it is an address-based list 
and includes no information about the residents at an address (or even 
whether it is a residence, rather than a business or unoccupied). This means 
that interviewers need to “screen” each selected address to determine 
whether any of its occupants are in one of the EM groups being sampled 
for the survey. While screening at PAF addresses is commonly used in sur- 
veys in the United Kingdom (e.g. screening for residents of a particular 
age or for households with children), this involves considerable extra field- 
work (and expense), especially when the group being screened for is only 
a small proportion of the population, as is the case with EM groups. The 
screening costs would likely be prohibitive for most surveys if it were not 
for a number of techniques that have been developed to increase cost-effec- 
tiveness. One important method for both reducing costs and increasing the 
efficiency of the sample is to design the sample so that areas with higher 
numbers of EM groups are over-sampled. A second technique for saving 
costs is the use of a screening technique called ‘focused enumeration’. 
While the effectiveness of these two techniques in reducing costs will de- 
pend on a number of factors, they have been successfully used in a large 
number of national surveys for sampling EM groups. These are described 
in more detail below. 


Over-sampling areas with higher than average expected screen-in rates 


If EM groups are geographically concentrated within certain parts of the 
country, as is the case in the United Kingdom for most EM groups, it is 
possible to reduce the screening costs by concentrating the sample in areas 
with higher strike rates — that is, areas with a greater proportion of EM 
groups. Of course, this approach requires having reasonable estimates of 
the likely strike rates in an “area”, which refers to an area that a single in- 
terviewer can work as a single assignment (in the United Kingdom, this 
usually refers to postcode sectors or electoral wards). However, data on 
EM populations in local areas within the United Kingdom is not easily ob- 
tained. Until the 2011 census, the best source available is 2001 census da- 
ta, which was nine years out of date at the time of this writing. The esti- 
mated strike rates in areas are therefore likely to be incorrect; while this 
will affect the efficiency of the selection procedure it should not bias the 
sample. 

The areas (say, electoral wards) are then divided into strata so that the 
strike rates will be similar within strata and will differ between strata. For 
a given total screening sample size, the optimum allocation per stratum (in 
the sense of maximising the effective sample size) is proportional to the 
stratum population size multiplied by the square root of the expected strike 
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rate (see Appendix B in Nazroo, Berthoud, Erens, Karlsen & Purdon 
2005). If more than one EM group is being screened for, the stratification 
and allocation process can become quite complex, as the aim is to come up 
with an efficient sample design for several EM groups simultaneously. 

An extreme example of over-sampling areas with high concentrations of 
EM groups can be found among Bangladeshis. As noted in the previous 
section, since Bangladeshis comprise 0.7 per cent of the population in 
England, it would be necessary to screen about 140,000 addresses to find 
1,000 Bangladeshis. However, census data show that Bangladeshis are very 
clustered geographically: 10 per cent of all wards include 89 per cent of 
the Bangladeshi population; and 20 per cent of wards include 95 per cent 
of Bangladeshis. Within the top 10 per cent of wards, Bangladeshis make 
up 2.6 per cent of the population, so 1t would be possible to reduce the 
screening exercise to 38,000 addresses if the sample was confined to those 
wards. The sample of Bangladeshis will be biased to the extent that the 11 
per cent of Bangladeshis not living in those wards differ from the other 89 
per cent, but the cost reduction is likely to make this bias tolerable. 


Exclusion of areas with very low strike rates 


Further cost savings arise if areas with very low concentrations of EM 
groups are excluded altogether from the sample. For example, about 94 per 
cent of the five main EM groups (Indian, Pakistani, Bangladeshi, Black 
Caribbean and Black African) in the United Kingdom lived in just 25 per 
cent of the wards in 2001. The strike rate for the other 75 per cent of wards 
is about 3 per 1,000. Excluding these 75 per cent of wards will introduce 
some bias into the sample, because EM residents living in predominantly 
White British areas will have no chance of selection. However, given the 
very high cost of screening, it may be judged more cost-effective to con- 
centrate screening resources on areas where the strike rate is higher and to 
live with the potential bias. 


Focused enumeration 


Even within those wards with higher concentrations of EM groups, the 
2001 census data suggests that about one-quarter of these wards will have 
a relatively low strike rate of less than 10 per cent. It is possible to improve 
the cost-effectiveness of screening in these areas by a technique known as 
focused enumeration (FE), which was developed in the 1980s by the 
National Centre for Social Research and the Policy Studies Institute 
(Brown & Ritchie 1981). It has now been successfully used to boost num- 
bers of EM groups in a large number of surveys in the United Kingdom 
(Smith & Prior 1996; Hales et al. 2000; Erens et al. 2001a; Erens et al. 
2001b; Hamlyn et al. 2003; Michaelson, Pickering, Wood & Scholes 2006; 
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Bolling et al. 2006). It is similar to a procedure used in the United States 
for sampling Black Americans in areas where they are a small percentage 
of local residents; the US procedure was termed ‘WASP’ for ‘Wide Area 
Sampling Procedure’ (Hess 1985, Jackson 1991). 

FE is a type of network or multiplicity sampling, which collects infor- 
mation not only about the selected household but also about others who 
are linked to the household in a specified way, such as relatives or close 
neighbours (Sudman & Kalton 1986; Kalton & Anderson 1986; Sudman, 
Munroe, Sirken & Cowan 1988). FE makes use of local knowledge by ask- 
ing selected households to identify members of EM groups at adjacent ad- 
dresses, so screening is carried out at an additional number of addresses as 
well as at the selected address. Usually, two addresses (sometimes this is 
extended to four addresses) to the right and left of the selected address are 
screened, and screeners are given standard rules to follow to identify 
neighbouring addresses for more complex dwelling layouts (e.g. blocks of 
flats or street corners). 

If the respondent at the selected address is certain that there are no EM 
residents in any of the adjacent addresses, the screener excludes these from 
the screening exercise (and up to nine addresses may be effectively 
screened for the cost of visiting one address). If the respondent is uncertain 
about the residents of an adjacent address, or positively identifies EM resi- 
dents, the interviewer will visit the address to complete the screening proc- 
ess. Evidence from the field suggests that FE works best in areas with a 
density of EM residents of perhaps 5 per cent to 10 per cent, but the exact 
cut-off for screening all addresses versus using FE will largely depend on 
cost considerations. 

While FE offers considerable cost savings, the method does have a num- 
ber of limitations. Firstly, when using FE, the strike rate for finding EM 
groups is considerably lower at adjacent addresses than at selected ad- 
dresses, perhaps by up to one-third less. This means that the issued sample 
size has to be larger than if full door-to-door screening was carried out, 
thereby offsetting some of the cost savings by using FE. But a bigger con- 
cern is whether this introduces any bias — for example, respondents of 
mixed ethnicity may be more likely to be missed. Another limitation is that 
FE works best when screening for people of Black or Asian origin; it can- 
not be used to screen for all EM groups or for particular EM subgroups 
such as recent migrants. Finally, FE is only suitable for face-to-face field- 
work. It cannot be used for telephone or postal surveys. 


Sampling ethnic minority groups: An example 


As is evident from the discussion above, designing a probability sample to 
select EM groups, while following standard sampling principles, is far from 
straightforward. The main steps that need to be followed are set out below. 
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First, set a minimum effective sample size for the main EM groups in- 
cluded (on the assumption that other EM groups will be selected as they 
come up, but with no minimum sample size target). For the purposes of 
this example, we have set a minimum achieved sample of 1,000 per main 
EM group, and the effective sample is 750. 

Second, calculate the expected screening-in rate for each main EM 
group in each area (e.g. ward). In the United Kingdom, this requires the 
use of census data, and the ONS definition of self-defined ethnicity. The 
farther away in time from the last census, the less accurate the strike rate is 
likely to be, leading to some inefficiencies in the sample design. 

Third, stratify the wards based on the estimated strike rates. Since the 
aim is to achieve a minimum sample size for each main EM group, there is 
not one strike rate per ward, but one per main EM group, so the creation 
of strata is multi-dimensional, and the rules for stratifying are unclear. In 
this example, five strata for each of the five main EM groups are set inde- 
pendently, and then cross-tabulated to give over 700 strata in all (which 
can be combined to make sample selection more manageable). 

Finally, allocate the sample across wards in order to achieve the minimum 
sample size requirements using a minimum number of issued addresses. 

Following these steps, an issued sample of about 45,000 addresses can 
achieve the target number of interviews based on the following 
assumptions: 

— 10 per cent of PAF addresses will be non-residential or unoccupied; 

— 90 per cent of households will answer the screening questions (yielding 
about 70,000 adults being screened); 

— all adults within a household who are from non-White EM groups will 
be screened-in; 

— 65 per cent of those screened-in will be interviewed. 


Based on these assumptions, Table 2.2 shows the achieved and effective 
sample sizes per EM group following this design. 

It is interesting to note from this table the relatively low number of 
achieved interviews with Chinese respondents. Despite being more numer- 
ous than the Bangladeshi group, the Chinese population is quite geographi- 
cally dispersed. Thus, the method of over-sampling areas with high strike 
rates is not suitable for boosting numbers of Chinese respondents. Similar 
considerations apply to the mixed ethnicity groups. For these EM groups, 
it is very difficult to use a PAF-based sample design which gives good 
coverage while avoiding very low strike rates. Other methods for boosting 
sample numbers of these groups need to be considered. 

This highlights one problem with this approach, which is its inability to 
efficiently sample particular EM groups such as the Chinese and mixed 
ethnic group populations. Another problem, already mentioned, is that this 
sampling approach is based on private residential households, so it 
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Table 2.2 Achieved and effective sample sizes using a stratified probability sample 


design 
Ethnic group Achieved sample size Effective sample size 
Indian 2,200 1,550 
Pakistani 1,800 1,200 
Bangladeshi 1,400 750 
Black Caribbean 1,250 900 
Black African 1,150 850 
Mixed: White/Black Caribbean 250 160 
Mixed: White/Black African 100 70 
Mixed: White/Asian 180 120 
Chinese 370 250 
Other non-White 1,250 850 


Source: Author's own elaboration 


excludes EM residents living in institutions — for example, bed and break- 
fast accommodation and hostels. Also, given the high costs involved in 
door-to-door fieldwork screening, data-collection costs are significantly in- 
creased for this sampling approach. 


Other sampling methods 


Other methods may be required for sampling particular EM groups or in 
particular circumstances. Ideally, probability sampling methods should still 
be used even if it means compromises in terms of coverage and the classi- 
fication system used. 

One alternative would be to generate samples for EM groups from exist- 
ing large surveys, perhaps accumulated over a number of different surveys 
(see Sudman & Kalton 1986). Of course, this requires that respondents’ 
consent was obtained on the initial survey to confirm their willingness to 
be followed up on in a subsequent, potentially unrelated, study; and ideally 
the follow-up will not be too long after the initial study in order to mini- 
mize the number of respondents who may have moved. It also assumes 
that the initial survey was high in quality, with a well-designed probability 
sample and a reasonable response rate. Any non-response biases in the ini- 
tial survey are automatically carried over to the follow-up study. But if 
such surveys are available, this approach can be a cost-effective way of 
generating samples of EM groups. 

An alternative is to generate such samples prospectively by adding 
screening questions to a current or planned survey. While this gets over the 
potential problems of out-of-date addresses and of obtaining consent to fol- 
low up, the other issues of following up a well-designed survey remain, 
and there is the added difficulty of having to wait to obtain the sample 
(Sudman & Kalton 1986). 
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An adaptive cluster design is another method with potential for sampling 
EM groups, providing the EM groups of interest are clustered within the 
relevant geographic level. At the first stage, a geographic cluster (e.g. a 
postcode sector or ward) is selected with probability proportional to the 
size of its population, and a number of addresses are selected within the 
cluster (e.g. two addresses per street) for screening for EM residents. If an 
EM resident is found at a sampled address, then additional addresses in the 
same street (e.g. neighbouring addresses) are also screened. At sampled ad- 
dresses where no EM residents are found, no further addresses in that street 
are screened (Thompson 1990; Kalton & Anderson 1986; Sudman & 
Kalton 1986; Elliott, Finch, Klein, Ma, Phoung Do, Beckett, Orr & Lurrie 
2008). A further adaptation of this approach, based on the use of ‘expert 
ratings’, has recently been tested for sampling Cambodian immigrants in 
California (Elliott, McCaffrey, Perlman, Marshall & Hambarsoomians 
2009). This relies on ‘experts’ to quickly classify addresses as likely or un- 
likely to contain an EM resident based on visible external ‘cultural indica- 
tors’. The aim is to significantly reduce costs by the use of these ratings in- 
stead of screening at all the selected addresses. 

Another probability method is to sample using surnames, although this 
has the drawbacks mentioned above. Chinese residents were sampled using 
this method in the Health Survey for England (Erens et al. 2001a, Sproston 
& Mindell 2006). For example, in 2004, all wards with more than fifteen 
Chinese residents (as identified in the 2001 Census) were identified and 
divided into two strata: those with 15-25 Chinese residents and those with 
26 or more. Ten wards were selected from the first stratum and 65 from 
the second, with probability proportional to the number of Chinese resi- 
dents. The ER for those wards was then scrutinised for households with 
Chinese-sounding surnames, and all households with at least one such sur- 
name were identified for screening. This sampling approach followed a 
methodology devised by the ONS, based on a list of the 1,300 most com- 
mon Chinese surnames appearing in the English version of the 1991 Hong 
Kong telephone directory. In all, 3,901 addresses were selected from the 
ER in these 75 wards, but even after sampling addresses which appear to 
contain Chinese residents only 492 households (12.6 per cent) were 
screened-in as eligible (e.g. because of people moving households, or be- 
cause some Chinese surnames — e.g. Lee — are also common English-lan- 
guage names). So not only are there biases built in to this method, but its 
success rate in identifying Chinese people is not particularly high. This ap- 
proach may work better in other contexts. For example, Elliott et al. (2008) 
found that lists of surnames to identify people of Hispanic ethnicity in the 
United States are able to achieve about 80 per cent sensitivity (i.e. 80 per 
cent of all Hispanics have surnames on the list) and have 90 per cent posi- 
tive predictive validity (ie. 90 per cent of those on the list identify as 
Hispanic). The same authors also suggest that surname lists can be 
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developed with equally good predictive value for Japanese, Chinese and 
Vietnamese residents in the United States. 

Another sampling method commonly used to boost EM (and other mi- 
nority) groups is network (or multiplicity) sampling. Network sampling re- 
lies on screening techniques, but in a more expansive way, so that informa- 
tion is obtained not only about the selected household for screening, but al- 
so about relatives, neighbours and others who may be connected to that 
household. As described above, focused enumeration is an example of net- 
work sampling, as information is obtained about neighbouring addresses at 
the same time as the selected address. Respondents selected using these 
methods need to be weighted in order for the sample to be unbiased, as the 
size of a person's network will influence the chances they will be sampled 
for the study. The need for weighting is one of the drawbacks of this ap- 
proach. Other drawbacks are the need for the network size to be correctly 
known, reliance on people to correctly identify others in the target group, 
potentially high costs and difficulties in tracing members of the network 
who have been identified, the loss of efficiency due to clustering within 
networks, and difficulties in estimating the size of the network. It also re- 
quires the group of interest to be well-networked, which may not apply in 
all circumstances. 

Most other methods of obtaining EM samples rely on non-probability 
sampling techniques. The most common method is ‘snowball’ sampling, 
which is frequently used for qualitative research, but also for surveys 
where there are tight time or cost constraints (Hughes, Fenton & Hine 
1995; Kahan & Al-Tamimi 2009). Researchers identify a small number of 
the EM group of interest and ask each of them to identify other members 
of the target group. These new members are then contacted by researchers, 
who in turn ask them to identify more group members. This continues until 
the target sample size has been reached. Snowball samples likely over-rep- 
resent people with a lot of connections, those who are closely linked with 
the initial “seed” respondents and people who are more likely to cooperate 
with surveys (Elliott et al. 2008). Since they are not probability samples, 
welghting cannot be used to adjust for this. Snowball samples are therefore 
likely to be “substantially biased” and the results “need to be assessed with 
considerable caution’ (Kalton & Anderson 1986). 

More recently, new developments with variants of snowball sampling, in 
particular respondent-driven sampling (RDS), have been shown to provide 
unbiased estimates, providing certain criteria are met. RDS uses the initial 
‘seeds’ as recruiters, referring other members of the target group to the 
study. Both the “seed” and the recruited member are given a coupon or 
payment for their effort. The recruited respondent then refers more mem- 
bers to the study. Referred respondents are volunteers, so no names or con- 
tact details need be passed to the researchers. A number of features distin- 
guish RDS from traditional snowball sampling. For example, ‘recruiters’ 
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are limited in the number of people they can bring into the study (usually 
to three or four), which results in longer chains and (hopefully) the recruit- 
ment of less connected members of the target group. Also, respondents are 
asked about the size of their network, as this enables selection probabilities 
to be calculated and the data to be weighted (so people with more connec- 
tions can be weighted down). In theory, RDS should continue its recruit- 
ment ‘waves’ until ‘equilibrium’ is reached, that is, until the distribution of 
some key respondent characteristics (such as age) become similar between 
waves. Although there is now a significant literature on RDS (Heckathorn 
1997, 2002; Magnani, Sabin, Saidel & Heckathorn 2005), and it is increas- 
ingly used for sampling hard-to-reach groups, some of its underlying as- 
sumptions remain untested and many studies do not adhere to its basic 
principles (Heimer 2005; Johnston, Malekinejad, Kendall, luppa & 
Rutherford 2008). 

Other methods involve sampling from a study population that differs 
from the target population but is easier to access. For example, facility- 
based sampling recruits respondents from facilities frequented by members 
of the EM group (e.g. Sin 2004 refers to sampling EM older people from 
community organisations). While convenient and easy to reach, it is clear 
that the people in touch with one, or even a range of, community organisa- 
tions will not necessarily represent the EM group overall. 

Time-space sampling is a related approach, involving first mapping and 
then selecting locations to sample (e.g. blocks, parks, mosques), and 
choosing segments of time in which to sample. Nonetheless, frequently, 
some members of the group will not visit such locations, so these methods 
can result in sampling bias (Sudman & Kalton 1986, Magnani et al. 2005). 


25 Data-collection issues 
Participation rates 


Response rates for surveys have been declining in many countries over the 
past two decades (Singer 2006, De Leeuw & De Heer 2002). In the United 
Kingdom, response rates declined for nearly all major government surveys 
in the 1990s (Table 2.3). 

For some UK surveys, response rates are lower for EM groups than for 
the White population, although there can be considerable variation between 
EM groups (Erens et al. 2001a; Elam, McMunn & Nazroo 2002; Health 
Education Authority 1994; Feskens, Hox, Lensvelt-Mulders & Schmeets 
2006). In the Health Survey for England 2004, compared with the response 
rate for the general population of 72 per cent, response rates ranged from 
56 per cent for Chinese respondents, to 60 per cent for Pakistani respond- 
ents, 62 per cent for Black respondents, 63 per cent for Indian respondents 
and 68 per cent for Bangladeshi respondents (Sproston & Mindell 2006). 
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Table 2.3 Survey response rates by year (percentages) 


Ethnic group 1991 1996 2001 
Labour Force Survey (wave 1) 84 81 78 
General Household Survey 84 76 72 
Family Expenditure Survey 70 62 62 
Health Survey for England 85 79 74 
Family Resources Survey = 69 66 
British Social Attitudes 67 68 59 


Source: Purdon and Nicolaas 2003 


Lower response rates among EM əroups may be even more of an issue in 
postal surveys. For example, a postal survey among hospital patients in 
England in 2005 achieved a response rate of 61 per cent among White re- 
spondents, compared with 41 per cent for Asian and 42 per cent for Black 
respondents (Sheldon, Graham, Pothecary & Rasul 2007). 

The higher rates of non-participation among EM groups are likely to be 
due to a number of factors including language barriers (for some groups), 
lack of trust, wariness of government authorities, perception that the re- 
search is unimportant or that their contribution is unimportant, reluctance 
to have their information written down, and a feeling that they have been 
over-researched (Groves & Couper 1998, Elam et al. 2002). 

Increasing response rates among EM groups is likely to require a num- 
ber of different strategies, including the translation of questionnaires and 
other survey materials, the provision of bilingual interviewers (or language 
helplines), stressing confidentiality to respondents, using mixed modes for 
data collection, emphasising the importance of the research for particular 
EM groups, explaining the value of the respondent's own contribution, and 
publicising the research, for example, through community organisations.’ 


Translation issues 


A proportion of people from some EM groups may not speak or be fluent 
in the source language in which the questionnaire was designed, so one of 
the most productive ways to increase response for these groups is to trans- 
late the questionnaire and survey documents into the appropriate 
languages. 

The process of preparing standardised translated questionnaires is quite 
complex and costly if it is done to robust standards. Moreover, for interview 
surveys it requires the use of interviewers/translators in the field who can 
speak and read the relevant languages. Allowing bilingual interviewers — or 
a bilingual member of the respondent’s household — to improvise their own 
translation of the survey questions will likely seriously compromise data 
quality because there would be a lack of consistency across interviews. 
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Therefore, the first issue to consider is what languages the questionnaire 
should be translated into. Given the diversity of EM groups, it is not practi- 
cal or cost-effective to cover all of the many languages spoken. A recent 
study among English school children found that 14 per cent had a first lan- 
guage other than English, and the children reported 240 different lan- 
guages. Which of these languages should be used for translation will de- 
pend on the EM groups included in the study and in what numbers, the 
proportion of each language group who cannot be interviewed in English, 
the number of interviews likely to be carried out in the language, the cost 
of each translation, as well as various political and equality issues. 
Previous surveys may provide a guide to help inform the decision. Thus, 
in the Health Survey for England 1999, the proportion of interviews carried 
out entirely in English was 85 per cent for Indians, 70 per cent for 
Pakistanis, 69 per cent for Chinese, and 34 per cent for Bangladeshis 
(Erens et al. 2001a). The languages offered on some of the large surveys in 
the United Kingdom which boost EM groups include Urdu, Punjabi (both 
Gurmukhi and Urdu scripts), Bengali/Sylheti, Gujarati, Hindi, Cantonese 
(simplified), Arabic (Egyptian), Somali (Latin script) and Welsh. National 
surveys in the United Kingdom which do not boost EM groups usually do 
not offer any translations, because the number of interviews in each non- 
English language will be too few to warrant the extra costs. For example, 
even though a high proportion of sampled Bangladeshi respondents may 
require a translated interview, there would be only 70 Bangladeshis 
sampled in a national survey of 10,000 individuals, and it may be judged 
that the cost of translation, and the complexity of recruiting and training bi- 
lingual interviewers in the sampled areas, is too great a use of limited re- 
sources for including perhaps 40-50 interviews. Of course, 1f the number 
of Bangladeshi respondents is to be boosted, then the need for a translation 
into Bengali/Sylheti becomes much greater. 

Next is the complex process of translating the questionnaire. Typically, 
this involves using a specialist translation agency to translate the source 
questionnaire, with the translation carried out by one individual and inde- 
pendently checked by another within the agency. Commonly, the translated 
questionnaire is then translated back into the source language to provide a 
basis for checking how well the concepts and questions translate into other 
languages while retaining the same meaning. Until recently, this technique 
of “back translation? was considered the gold standard for translating sur- 
vey instruments, and it is still commonly used, for example, for the 
European Quality of Life Survey (2009). 

Recently, however, much more attention has been paid to this process 
and the problems that can arise from poor translations (Harkness, Pennell 
$ Scoua-Glusberg 2004). Team or committee approaches to translation are 
becoming more common. For example, the European Social Survey now 
use a process called Translation, Review, Adjudication, Pre-testing and 
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Documentation (TRAPD) that involves a team approach with translators 
and survey experts working together to produce the translations (Harkness 
2008). 

In order to shorten the period required for the translation process, the 
United Kingdom's Understanding Society survey uses a modified TRAPD 
approach that involves up to four stages. First, an agency translator trans- 
lates the questionnaire. Then a second translator from the same agency re- 
reads the text, comparing the English source text with the translation to en- 
sure there are no grammatical mistakes, typographical errors, mistransla- 
tions or omissions, improving the style where necessary. Third, an inde- 
pendent checker (i.e. someone who does not come from the same agency 
as the first two translators), reviews the translation and adds comments on 
the accuracy, completeness, interpretation and consistency of meaning with 
the English questionnaire. If the checker is satisfied with the translation as 
it stands, they can sign off on it. A fourth stage only arises if the checker 
has made comments or suggestions on the translation; it then goes back to 
the original translator, who reviews the comments and incorporates the 
suggestions if they are judged to be appropriate. The original translator 
then signs off on the translated questionnaire. 

All the translation work on Understanding Society is coordinated 
through the Language Management Utility (LMU), a web application 
which was originally developed by CentREdata at the University of 
Tilburg for the Survey of Health, Ageing and Retirement in Europe 
(SHARE).* 

Other team approaches to translation are possible, such as that outlined 
by Bhopal et al. (2004), who suggest comparing each language with every 
other translated language (as well as with the source language) and carry- 
ing out tests of validity, reliability and responsiveness in each language. 

Another recent suggestion has been the use of anchoring vignettes to as- 
sist comparisons between different cultural groups which may interpret 
identical questions in different ways. This technique requires respondents 
to react to scenarios that describe hypothetical situations, aspects of which 
are varied in a controlled way, in order to examine different dimensions of 
the concept being investigated. Vignettes have been used in a number of 
cross-national studies, including wave 2 of SHARE and in studies of self- 
rated health and political efficacy (King, Murray, Salomon & Tandon 
2003; Salomon, Tandon & Murray 2004). The aim is to show how differ- 
ences in responses between groups, for example, on self-rated health, are 
partly due to real differences in health and partly due to variations in inter- 
preting the scale. The idea of the vignettes is to provide an anchor which 
can be used to adjust the self-assessments. Salomon et al. (2004: 258) con- 
clude that ‘[a]nchoring vignettes can provide a useful tool for standardizing 
perceptions of health and adjusting self-reported measures to account for 
variation in norms and expectations for health’. 
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Use of bilingual interviewers 


While the preferred option is to have translated questionnaires administered 
by trained and experienced bilingual interviewers, this is not always feasi- 
ble or the most cost-effective approach — for example, in an area where on- 
ly a handful of people are likely to require an interview in a particular 
language. 

A second option, then, is to have a bilingual professional interpreter ac- 
company the English-speaking interviewer on the visit in order to read out 
the questions to the respondent and interpret their answers for the inter- 
viewer. While not trained as a survey interviewer, it is recommended that 
the interpreter receive some training in interview techniques and be given 
an overview of the aims of the study and of the questionnaire. This option 
is clearly costly, since it not only requires two people to be present during 
the interview, but also professional interpreters are quite expensive. So the 
costs of this approach have to be balanced against the costs and practical- 
ities of recruiting bilingual interviewers. 

This is the approach adopted in the Understanding Society survey. 
Moreover, since the questionnaire is not being translated into all languages 
that may be encountered in the field, if a language is spoken which is not 
translated, another adult household member is permitted to translate for the 
respondent. When interviews are done in translation, this needs to be docu- 
mented with the data. 


Ethnic matching 


While language matching may be crucial for including in the survey non- 
English speaking members of some EM groups, the evidence for matching 
interviewers and respondents by ethnicity is mixed. Ethnic matching may 
affect different stages of the survey process. For example, ethnic matching 
may help to improve overall response rates among EM groups, by increas- 
ing the levels of trust, legitimacy and credibility in the survey, and poten- 
tially mitigating perceptions that the study is irrelevant. However, the ex- 
tent to which ethnic matching has a positive impact on response could de- 
pend on the survey topic. For some sensitive topics, such as racism, ethnic 
matching has been shown to be helpful (Elam et al. 1999). However, there 
is also evidence that ethnic matching has no effect on most topics 
(Aspinall 2001, Rhodes 1994). It may even have a negative effect, for ex- 
ample, when discussing a taboo topic, and confidentiality worries may 
arise when being interviewed by a member of what may be a relatively 
small community (Phoenix 1994, Grewal & Ritchie 2006). 

For similar reasons, ethnic matching may affect responses given to sur- 
vey questions. By increasing rapport between interviewer and respondent, 
the latter may be more willing to give ‘honest’ answers on certain topics. 
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In an opinion poll in the United Kingdom conducted soon after the US/UK 
military strikes began in Afghanistan, British Asians were more likely to 
voice opinions contrary to the “perceived national mood” if the interviewer 
was also Asian (although this was not consistent for all questions in the 
survey) (Worcester 2002). But there is also the contrary argument that eth- 
nic matching may inhibit ‘honest’ reporting of culturally taboo behaviours 
(e.g. if a Muslim interviewer asks a Muslim respondent about alcohol 
consumption). 

Ethnic matching also raises practical difficulties in the field. Examples 
are increased costs (if there is a need to recruit and train EM interviewers), 
more complex fieldwork arrangements, and possibly an extended fieldwork 
period. 

Given the difficulties of ethnic matching, it is not often implemented in 
large-scale UK studies. What is emphasised instead is the need for inter- 
viewers to be fully aware of general cultural issues (e.g. forms of greeting 
and acceptable dress and behaviour in people’s homes) as well as any po- 
tential sensitivities in the questionnaire for particular EM groups. If possi- 
ble, interviewers, and researchers, should be trained in cultural awareness 
and sensitivity (Papadopoulos & Lees 2001). This may raise the need for 
gender matching in some circumstances, such as for Muslim women who 
cannot be alone in the same room with a non-family member male 
interviewer. 


2.6 Conclusions 


All surveys involve making compromises between what researchers would 
like to achieve and what is practical within budget and time constraints. 
These issues are amplified when it comes to research focusing on EM 
groups, which present unique challenges. This chapter showed some of the 
key issues that need to be addressed when designing EM surveys in the 
United Kingdom, and gave an indication of the types of compromises that 
are often made, while trying to minimise the impact on data quality. 

For example, even well-resourced national surveys have to accept limita- 
tions in the EM groups that can be sampled in sufficient numbers to allow 
separate analysis. These limits could be due to an EM group being rela- 
tively rare within the population, or it could be that there are no reliable 
data identifying the geographic areas that contain a relatively high percent- 
age of a particular EM group. Or the data may show that some EM groups 
are dispersed throughout the country, which makes sampling difficult and 
fieldwork costly. Researchers are also often constrained to working within 
an existing ethnic classification system for practical reasons (such as locat- 
ing areas of residence of particular EM groups using census data), even if 
the categories defined by that system are not a precise match to the study’s 
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population of interest (e.g. using ethnic groups as a proxy for language or 
religious groups). 

If an EM group included in the survey is likely to lack a solid basis in 
the national language, then decisions are required regarding translations. 
These considerations have cost implications. Not least there is the need to 
decide which languages the questionnaire and survey materials should be 
translated into, given the time required to carry out each translation, the 
cost of doing so, the need to recruit bilingual interviewers or interpreters, 
and the potential complexities of organising fieldwork so that the bilingual 
interviewers/interpreters are sent to the appropriate households. 

The challenges involved in carrying out EM surveys often lead to ‘cut- 
ting corners’ in ways that would be considered unacceptable on standard 
surveys of the general population. This is particularly the case with regard 
to sampling EM groups, with surveys too readily opting for non-probabil- 
ity methods because of the difficulties or costs involved in obtaining a 
probability sample. Questions may be justified concerning the results of 
many surveys among EM groups because of their use of dubious methods. 
This chapter attempted to describe methods for surveying EM groups to 
produce robust results even though some compromises in study objectives, 
design or practice may be required. 


Notes 


The ethnic identity question was again revised by ONS for the 2011 census. 

Note that ONS produces these figures for England only, not for the whole UK. 

More up-to-date data from the 2011 census is now available on the ONS website. 

Since the percentage of EM groups is lower in Wales, Scotland and Northern Ireland than 

in England, for surveys covering the whole UK the numbers of EM respondents will be 

even lower. 

5 For more information on this see www.understandingsociety.org.uk/design/sample/ethnic. 
aspx. 

6 There is a considerable literature on network sampling, including Sudman and Kalton 
1986, Sudman et al. 1988, Kalton and Andersen 1986, and Elliott et al. 2008. 

7 A list of practices that have been used or considered as ways of increasing participation 
among migrants in labour force surveys carried out by EU member states may be found 
in Barnes (2008). See also chapters 7 and 8 in this book. 

8  www.share-project.org; www.centerdata.nl/en/TopMenu/Projecten/SHARE/ 
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3 The 2oo7 Spanish National Immigrant Survey 
(EN!'): Sampling from the Padrón 


Ignacio Duque, Carlos Ballano and Carlos Pérez 


3.1 Introduction? 


International migration flows to Spain have been intense in the last fifteen 
years. While at the start of the 1990s, the share of foreigners with respect 
to the total population was barely 1.5 per cent, by 2000 it was 2.3 per cent, 
and by 2009 it was 12 per cent. The Spanish immigrant population’s rate 
of increase has been remarkable compared to other countries as well 
(Cebolla & González-Ferrer 2008: 12). While in 1990, Spain was not even 
among the twenty countries in the world with the highest immigration 
rates, by 2005 1t was already tenth in the ranking (in absolute figures). 

Although the Spanish statistical system had different registers and sur- 
veys that could be used to learn more about this new population, there was 
no single source of specific data with information on the trajectories of im- 
migrants, previous countries where they had lived, housing, social and 
family networks, and so on. The Instituto Nacional de Estadística (INE, 
Spanish National Statistics Institute) thus decided to carry out the National 
Immigrant Survey (END) in order to fill this information gap and to satisfy 
the demands for data on this new population group (see INE 2009). 

The literature on ethnic and migration studies played a key role in the 
ENI’s design. Aims, definitions, operational variables, categories and clas- 
sifications are the cornerstones of any production of social data, including 
sample surveys. In social surveys, data production is restricted by the so- 
cial categories and practices specific to any society and historical point in 
time (Gigerenzer, Switjtink, Porter, Daston, Beatty & Krüger 1989; 
Desrosiéres 1993; Porter 1995). This relationship between tools and society 
applies to the statistical techniques themselves as well, as many historians 
of statistics have illustrated from the beginning of the Bielefeld School un- 
til today (Stigler 1999). 

The ENI was based on the idea that immigrants should be defined in ob- 
jective terms, that is, as people who have a particular experience of migra- 
tion, rather than in subjective terms, according to self-identification with a 
cultural or ethnic group. In our view, it would not be suitable to choose an 
approach to the definition of the immigrant in Spain based on a self- 
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defined concept of ‘group membership’. We preferred targeting the survey 
to measure empirical experiences of changes of usual residence (Spain 
being the destination country). This decision does not deny the importance 
of categories that cannot be directly measured or included in questions, 
such as subjective social class and ethnic identification. Many of these are 
among the most important analytical tools in social sciences. In any case, 
this definition of migration leaves researchers free to choose or combine 
any of the categories from the list of variables available in the ENI (more 
than 1,500). 

The Spanish statistical infrastructure was crucial for the development of 
the ENI. This is a point we wanted to stress by including the ‘Padrón’ 
(population register) in the title of our chapter. The statistical infrastructure 
is like other social, economic or life pre-condition: it is taken for granted 
until there is a problem. Having a continuous population register such as 
the ‘Padrón’ makes a huge difference when designing a survey like the 
ENI, for example, in terms of sampling design, since it can be used as a 
sample frame. (In contrast, no such register was available in the UK case 
highlighted in chapter 2 of this volume.) 

After a brief introduction on the way the Spanish statistical system deals 
with the consequences of international migrations, this chapter describes 
the history and main traits of the ENI, especially the decisions made con- 
cerning the sample and the organisation and results of the fieldwork. 


3.2 International migrations and the Spanish statistical system 


From 1960 to 1983 there was huge growth in Spanish international migra- 
tions (leaving Spain), together with extremely heavy internal migration asso- 
ciated with the process of urbanisation, industrialisation and departure from 
agriculture and the rural world. During this period, the statistical system de- 
veloped a modest instrument to track these migrations, taking advantage of 
the fact that the flows to Germany, France and Switzerland were derived from 
intergovernmental agreements. As of 1961, statistics of flows to these coun- 
tries were supplemented with data from trans-oceanic assisted migration.’ 

During the second half of the 1980s immigration to Spain started to de- 
velop, but for more than a decade the quantitative relevance of this new 
population was small.* When, in 1992, the first National Statistics Plan 
was passed, its layout was still quite similar to that of the previous decade 
and took no particular notice of the new phenomenon of incoming interna- 
tional migration. Before the statistical plan of 2005-2008, no major signs 
of change were in evidence regarding the effective production of statistics 
especially designed to measure migration. 

The ‘Padrón’ is built from municipal population registers, which are ad- 
ministrative registers that keep a record of all inhabitants of each 
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municipality. Luckily, in 1996, before the immigration shock came about, 
the Spanish statistical system had already designed and implemented fun- 
damental improvements to the “Padrón”, for example, making it a continu- 
ous population register. Before these changes were implemented, the mu- 
nicipal population registers that provided the basis of the ‘Padrón’ were 
completely redone every five years by field agents who collected informa- 
tion dwelling by dwelling. In the current system, started in 1996, coordina- 
tion of municipal registers is achieved through monthly exchanges of infor- 
mation between the municipalities and the INE. Municipalities communi- 
cate changes in the register in the previous month, and the INE is in 
charge of checking for discrepancies and double-counts when combining 
all of the information into the national list. Key advances in the new sys- 
tem were thus comprehensive computerisation, creation of a standardised 
system of exchange, and greater central coordination and oversight of the 
whole process by the INE. 

Making the ‘Padrón’ a continuous population register was a fundamental 
step towards raising the quality and consistency of information and making 
more intensive use of administrative data. There are ambitious plans to en- 
hance its effectiveness further:? making it an integrated register with great- 
er online functionalities for documenting residency, making quality and 
non-duplication checks easier, incorporating a repertory of territorial units 
and dwellings linked to GIS systems and other government bodies, among 
other improvements. 

For the purpose of this chapter it is important to note that the ‘Padrón’ 
includes both authorised and non-authorised foreign residents.° 
Registration is needed for immigrants to have access to the public health 
and education systems (irrespective of whether they have a residence per- 
mit). In 2003 a new regulation was approved to improve the accuracy of 
the information on foreign migrants. This requires foreigners from coun- 
tries other than EU member states to renew (or ‘confirm’) their registration 
in the population register every two years. Otherwise they would be re- 
moved from the register. 


3.3 Role of the ENI in supplying information on international 
migrations in Spain 


History of the project 


The ENI project originated in 2004, when a group of researchers and aca- 
demics from Spanish universities” proposed the undertaking to the INE 
and Ministry of Labour. This same group contributed substantially to the 
preparation of the questionnaire and project. The cost for all phases (sam- 
pling, elevation, field-work and data filtering) was covered by the INE, 
which applied the same working systems, tools and quality standards as it 
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does for any of its statistical operations. The participation of this academic 
team in some tasks and phases of information production (mainly the ques- 
tionnaire and the basic report for presenting the results, but also to define 
the edits and checks of the questionnaire) was considered highly positive 
as it combined INE regular procedures with expert academic knowledge. 

Consultations on the project took place among numerous institutions 
and researchers, who contributed proposals and interesting reviews. This 
permitted the production of information to be more suited to the require- 
ments of the analysis. In any event, special care was taken to ensure that 
the academic goals would also accommodate the general interests of all 
types of users. 


Research on the characteristics of migrants, their family relationships and 
their social networks 


Ascertaining outside migratory flows is undoubtedly a basic priority of the 

Spanish statistical system. Essential for this purpose are the statistical ex- 

ploitation of the “Padrón” and statistics on residential changes. The ENI is 

not located at this first and crucial level, since its role is not to research the 
basic magnitudes of the flows. Instead, the ENI studies aspects of reality 
that administrative records (such as the “Padrón”, the register of residence 
permits, the Civil Register and others) do not reveal, or reveal only with 
great difficulty. The main goals of the ENI are five: 
investigating the socio-economic characteristics of immigrants that de- 
termine their experience and insertion into the labour market, such as 
education and occupation (in their country of origin and in their itiner- 
ary through Spain); 

— investigating complete migratory paths from the starting point as well 
as migratory experiences in other countries and within Spain; 

— gathering information on the housing situation of immigrants, including 
physical characteristics, occupancy status and relationship with those 
with whom they live; 

— gathering essential information about the present relationship of immi- 
grants with their country of origin; 

— obtaining data on the relationship of immigrants with their host country 
(Spain) in different civil, social and cultural aspects. 


The ENI also aims to go beyond the single individual as the object of re- 
search. There is no doubt that in the case of vital decisions, such as those 
related to international migration, individual characteristics are likely to be 
key explanatory factors. Nevertheless, most research sees migration as a 
project in which families, groups and even entire societies and regions play 
an essential role. For this reason, the ENI intends to 1) collect data about 
the basic characteristics of all the residents in the home of the selected 
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person and 2) provide details of the family relationships in which each per- 
son is involved, including parents, brothers and sisters, spouse and chil- 
dren, and to do this even if these family members and spouses are not liv- 
ing under the same roof, and regardless of where they are located. 

In spite of its panoramic nature, the ENI has consciously left certain im- 
portant aspects out of the scope of the research. It does not aim to obtain 
an accurate measurement of the costs associated to each of the aspects 
linked to the migratory process, as this would have required a different 
strategy. Nor has it attempted to assess the budget and expenditures of the 
families and the households with immigrants, as it made no sense to set up 
a parallel line of work to that of the Family Budget Survey. Even with 
these limitations, the scope of the survey is enormous. 


3.4 Methodology 
The target population 


Being the first national immigrant survey, the main objective was to get a 
general picture of immigrants in Spain. However, adopting a concrete defi- 
nition of the target population raised some questions: did the survey have 
to be sufficiently broad as to encompass the entire national territory? 
Would it be addressed to foreigners or to those born abroad? Did it need to 
include foreign/foreign-born individuals from all countries? Did it have to 
include all immigrants irrespective of the length of time they had lived in 
Spain? Should it include the descendants of immigrants? Clearly, no defini- 
tion can cover all objectives for all potential users, and each definition en- 
tails limitations too. In addition, the definition of the target population 
adopted had to be practical and allow for efficient selection of people using 
the available sample frames. 

The main decision was the criterion the survey would use when defining 
the target population: nationality or country of birth. Choosing (foreign) na- 
tionality as the main criterion meant leaving out of the survey immigrants 
who had obtained Spanish nationality, as well as the descendants of Spanish 
people who had emigrated to other countries, mainly in the 1960s, who had 
typically been born in foreign countries but held Spanish nationality. 
Conversely, choosing country of birth as the main criterion implied leaving 
out of the analysis the so-called “second generation” of immigrants (children 
born of immigrant parents in Spain). Considering that the presence of a sec- 
ond generation is still limited in Spain, as a relatively young immigration 
country, and with the additional fact that the second generation has not had 
the experience of migration which should be associated with the target pop- 
ulation, this did not seem to be a problem. The target population was finally 
defined as individuals born abroad, sixteen years of age or older and resident 
in Spain for at least a year, or having resided less than a year but with the 
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intention to stay, excluding people born abroad who held Spanish nationality 
and had moved to Spain before two years of age. 

In spite of the novelty of this criterion within the Spanish statistical sys- 
tem, it seemed from the first instance that the population born outside 
Spain was the relevant group. In the current context of international migra- 
tions to Spain this definition of the research universe had a number of ad- 
vantages: 1) it is rooted in the demographic approach to migration, beyond 
nationality; 2) it is based on a clear, well-known variable, which does not 
change over people's lives and is available in the basic population reper- 
toires that are required for empirical research; 3) it is consolidated in inter- 
national standards and allows for comparisons among many countries; 4) it 
permits inclusion in the target population of significant flows of recent im- 
migrants who have acquired Spanish nationality. 

As already noted, this definition has shortcomings as well,!° but it 
seemed to be the most appropriate given the degree of maturity of the 
flows in Spain’s migratory system. 


Pilot tests 


Since this was the first time that the INE had conducted a survey addressed 
exclusively to the immigrant population, it was considered necessary to 
carry out several tests in order to investigate aspects of the survey design. 
A cognitive test for the questionnaire and for the introductory letter for the 
survey was also carried out, although this chapter concentrates on the re- 
sults of the pilot tests. 

The sample of the first pilot test consisted of fifty sections, with ten tar- 
get subjects and ten substitute subjects per section. The persons selected 
from the population register had to meet two prerequisites: being sixteen 
years of age or older and having been born in a foreign country. This sam- 
ple was obtained by a non-random method, through the choice of three 
provinces that were considered representative of the principal groupings of 
immigrants: Madrid, Malaga and the Balearic Islands. The immigrant pop- 
ulation was divided into five groups of one hundred persons each, accord- 
ing to the location of their country of birth: Eastern Europe, Central and 
South America, the European Union, North Africa and the Rest of Africa. 
Problematic cases in the target sample were replaced by using the substi- 
tute sample. Such replacements had to be made within a given section and 
be taken from the same group of nationalities. At the end of the question- 
naire a section was included requesting identification data for all residents 
of the dwelling (first name, surname and identity document number). 

There was in fact a very large number of non-locatable persons, mainly 
because the person selected had moved to another address. This high per- 
centage rendered unfeasible the sampling procedure by which persons were 
selected directly from the “Padrón”. Nevertheless, it was observed that in 
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the majority of dwellings visited that turned out to have a non-locatable 
person, there was at least one person who had been born outside Spain. 
This suggested the possibility of following a new sampling procedure, 
which was subsequently used in both the second pilot test and the defini- 
tive survey, based on selection from the ‘Padrón’ of a sample of dwellings 
where there was at least one foreign-born individual. Therefore, the first pi- 
lot test produced evidence of a need to reorganise the procedure for select- 
ing the sample for the ENI. 

The number of non-contact units was very high in one of the provinces 
covered by the pilot test, due to the registration of persons from the 
European Union who spend long periods of time in their country of origin. 
The number of refusals was not especially high. Another significant find- 
ing, in view of the implications it would have for the organisation of the 
fieldwork for the definitive survey, was that few cases were found in which 
it was not possible to conduct the interview due to language problems. 

With respect to the questionnaire, a reluctance was found to reply to the 
questions that dealt with brothers and sisters who did not live in the same 
dwelling. Similarly, in view of the heterogeneity of the target population, 
certain questions were found to be inappropriate for some immigrant 
groups. It was also observed that the word ‘immigrant’ (in the title of the 
survey) had pejorative connotations for some groups, such as those persons 
who had lived in Spain for many years and those originating from devel- 
oped countries. However, it was not possible to change the name of the 
survey, since the originally chosen name had already been included in the 
National Statistics Plan. 

Once the procedures had been redesigned on the basis of the findings of 
the first pilot test, a second pilot test was conducted, the principal aim of 
which was to analyse the viability of the new selection procedure. For this 
purpose, in the same provinces chosen for the first pilot test, fifty sections of 
ten target dwellings and ten substitute dwellings each were selected. The 
dwellings were chosen on the basis that they included one or more persons 
who had been born outside Spain (and were registered in the ‘Padrón’). 

During the course of the fieldwork a list with the members of the dwell- 
ing was built up, and an individual was selected to be interviewed by using 
a Kish method. Findings confirmed the observations of the first pilot test: 
in the vast majority of the dwellings selected there lived representatives of 
the population who were the subject of the survey, which consequently 
confirmed the new selection procedure. Findings also served to determine 
the definitive working ratios. 


Sampling design 


The main purpose of ENI sample design was to provide estimations, at the 
national level, of the main characteristics of the population born abroad, 
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sixteen years of age and older and resident in Spain at the time the survey 
was conducted. With this aim in mind, as already mentioned, the first idea 
was to select a sample of persons using as a frame the most updated popu- 
lation register (“Padrón”) available at the start of the fieldwork. 
Nevertheless, due to the great difficulties encountered during the first pilot 
test, it was decided that the selected sampling unit would be the dwelling 
and then all persons in scope and residing there would be investigated. 

The sampling frame used was the “Padrón” with reference to September 
2006, the closest possible to the time of the fieldwork. Although the main 
target was to obtain national-level estimations, the sample design consid- 
ered an independent sample for each Autonomous Community of Spain, in 
order to obtain estimations at the regional level for the most important na- 
tionalities. Eight groups of foreign-born individuals were included for this 
purpose, mainly based on their numerical presence: 1) Ecuador, 2) 
Morocco, 3) Romania, 4) Latin America (without considering Ecuador), 5) 
Africa (without South Africa) and Asia (without Japan and other developed 
countries from the area), 6) North America (without Mexico) and Oceania, 
7) European Union (fifteen member states) plus those born in European 
Economic Space countries that are not EU members (Switzerland, 
Liechtenstein, Norway and Island), and 8) those born in any other country 
of the world as the final group. 

With the aim of meeting the survey objectives, two different and inde- 
pendent sample types were considered: 1) sample A, made up of dwellings 
with at least one foreign-born resident, and 2) sample B, made up of dwell- 
ings with only Spanish-born residents (none foreign born). 

Eligible interviewees should in principle live in dwellings from sample A, 
but due to the high geographical mobility of the group, it was important to 
check for their presence in dwellings belonging to sample B. In both cases, a 
three-stage random sampling design was used with stratification of primary 
sampling units (PSUs). The PSUs were census sections (geographical areas), 
the secondary sampling units were main family dwellings, and finally, the 
third-stage sample unit was one person selected among the foreigners living 
in the dwelling. In the PSUs selected for type A samples, only type A dwell- 
ings were investigated; the same applied for type B. In some cases, the same 
PSU (census section) was selected for the two types of samples. 

The PSUs (census sections) were grouped into strata according to the 
size of the municipality to which they belonged. The definition of the strata 
took into account the target population distribution and the sample size of 
each type of sample, so the stratification was different in each sample. In 
type A samples, six strata were considered, and only three were considered 
in type B samples. Within each stratum the sections were grouped in clus- 
ters or substrata using a hierarchical cluster analysis performed by the 
Ward algorithm programmed in SAS. The following information from the 
sections was used to make the clusters: 
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— for the type A sample (dwellings with foreign-born resident), percen- 
tages of foreign born for the eight nationalities considered; 

— for the type B sample (dwellings without foreign-born occupant), per- 
centages for each group by sex and age. 


In the sample of dwellings with a foreign-born occupant (sample A), PSUs 
(census sections) were selected with probability proportional to the number 
of foreigners aged sixteen years or older and living in the census section. 
Within each selected section, ten dwellings were chosen with equal proba- 
bilities, using a random systematic sampling. In order to achieve good rep- 
resentativeness of the different nationalities in the selected section, dwell- 
ings were sorted by the predominant nationalities in the census section be- 
fore drawing the dwellings. In the sample of dwellings with only Spanish 
residents (sample B), PSUs were selected with a probability proportional to 
the number of dwellings with only Spaniards living in them. Within each 
selected section, ten dwellings were chosen with equal probabilities and us- 
ing systematic random sampling. Before drawing dwellings, they were 
sorted by their size. 

The third-stage sample unit was randomly selected among the foreign- 
born residing in the dwelling. In order to do this, the interviewers were in- 
structed to set up a list with all of the foreign-born individuals in the dwell- 
ing, and to select one of them using the Kish methodology. 

To achieve the survey targets, the initial sample size was about 21,000 
dwellings distributed among 2,100 sections. Later, the regions of Murcia, 
Navarre and the Balearic Islands decided to enlarge the sample size in their 
areas with the objective of providing additional disaggregated regional esti- 
mations. The final sample size was 2,270 sections, of which 1,770 were of 
type A (with foreigners) and 500 of type B (only Spaniards). 

The sample size in each region was calculated following a mixed proce- 
dure of uniform and proportional allocation, the latter according to their 
size (measured by the number of foreigners and by the number of dwell- 
ings without foreigners, respectively, for sample type A and B). The sam- 
ple allocation among provinces and strata is strictly proportional to their 
size, as indicated above.'' 


35 Analysis and results of the fieldwork 
Organisation of the fieldwork 


The collection of data was organised around 33 geographical areas corre- 
sponding to the sub-national offices of the INE. There are fifty provinces 
in Spain (plus the cities of Ceuta and Melilla), so in some cases the same 
sub-national office was in charge of organising the field-work allocated to 
two provinces. 
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A survey inspector was appointed in each of these areas. Survey inspec- 
tors attended a three-day training session. They were in charge of training 
the interviewers and their supervisors. One supervisor was designated for 
every three interviewers. Two persons were hired to conduct interviews in 
English, Arabic and French." 

Interviews were carried out face-to-face, though telephone interviews 
were conducted in a few cases when they had to be done in one of the for- 
eign languages just mentioned. Data collection started on 30 October 2006 
and lasted until 22 February 2007. Six working days was set as the time 
slot necessary for a type A section, and three for a type B section. 
Computer-assisted personal interviewing (CAPI) was used as the collection 
mode for the personal interviews with a view to avoiding flow errors, in- 
cluding coherence checks during the filling in of the questionnaire. 
Interviewers and supervisors were equipped with mobile telephones for 
ease of communicating with inspectors and with interviewees. 


Results of the fieldwork 


Overall results 
The final response rate was 67.4 per cent (Table 3.1). About a third (32.6 
per cent) of non-responses (19.4 per cent of the total sample) was due to 
non-contact (of which 15.4 per cent were non-contact with anyone in the 
dwelling and the remainder non-contact with the person selected to be in- 
terviewed). Another 11.8 per cent of non-responses was due to refusals 
(10.6 per cent being refusals by the first person contacted in the dwelling 
and the remainder refusals by the selected person). The remaining 1.4 per 
cent was due to incapacity to reply, essentially caused by language prob- 
lems. A total of 15,465 valid questionnaires resulted from the 32,541 type 
A dwellings that were visited during the fieldwork. 

The effective sample obtained, consisting of target plus substitute sam- 
ples, was 87.4 per cent of the theoretical sample. The in-scope rate in tar- 
get dwellings in type A sections was 75.2 per cent of dwellings in the 


Table 3.1 Main results of the fieldwork 


Outcome Percentage Detailed reasons 
Not at home 19.4 15.4 (dwelling non-contact) 

4.0 (non-contact of selected person) 
Refusals 11.8 10.6 (refusal of the group) 

1.2 (refusal of the selected person) 
Incapacity to answer the survey 1.4 1.1 (incapacity of the group) 

0.3 (incapacity of the selected person) 
Response rate 67.4 
Total 100 100 


Source: INE 
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survey, the percentage of dwellings where no target population lived was 
10.8 per cent and 14.0 per cent were empty and non-principal dwellings. 
In the case of target dwellings in type B sections, the in-scope rate was 0.8 
per cent of dwellings in the survey, 72.9 per cent were dwellings at which 
no target population lived, and 8.6 per cent were empty and non-principal 
dwellings. The remaining 17.7 per cent represented problems that pre- 
vented the qualification of the dwelling as either eligible or ineligible for 
inclusion in the survey. The incorporation of type B sections allowed us to 
verify that the number of cases of eligible interviewees living in these 
dwellings was extremely small and, as a result, that the sample frame used 
included almost all the objective universe. 


Differences in fieldwork results among immigrant groups 

According to the reports provided by interviewers and supervisors, three 
groups of target population can be identified from fieldwork results and re- 
spondents” behaviour regarding the survey. 

The first group is composed of individuals born in a developing country 
who come to Spain to seek economic or social conditions better than those 
in their country of origin. The persons belonging to this group were, in 
general, willing to cooperate. In many cases they showed an interest in par- 
ticipating in the survey, or they felt ‘obligated’ to participate in it. 
Nevertheless, some in this group were reluctant. The two main problems 
regarding participation were difficulty in finding a time slot to do the inter- 
view, since they tended to leave home early in the morning to go to work 
and return home late, and their reluctance to provide information about 
people in the dwelling who were not authorised to live in Spain. 

The second group is made up of people who were born in a developed 
country and chose Spain as the country in which to “enjoy their free time”. 
Most are retired people from other countries of the European Union or, in 
fewer numbers, from the United States. They usually live in holidaymaking 
areas such as along the Mediterranean coast and the Spanish islands. They 
live in widely-scattered areas, and are in many cases difficult to identify. 
Often, these persons divide their lives between their country of origin and 
Spain, which makes establishing contact with them difficult. They do not 
consider themselves to be ‘immigrants’ and were reluctant to reply when 
they heard the name of the survey. They expressed surprise when asked to 
answer certain items in the questionnaire (such as, e.g. how much their 
journey to Spain had cost, arrangements for sending money to their home 
country and the cost of living). 

Finally, the third group is mainly composed of individuals born outside 
Spain, with at least one of their parents having Spanish nationality. Most 
of these people also have Spanish nationality. The persons in this group 
were reluctant to reply or were ill at ease because they did not think they 
should be included in the survey. Cases of Spaniards who had been born in 
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former Spanish overseas territories were the most problematic ones. Given 
that they hold Spanish nationality, in many cases they found it offensive to 
be considered part of the target population of the survey. 

A final expected problem was the existence of dwellings where various 
immigrant family groups lived, without being related to one another. The 
results obtained in the second pilot test indicated that these problems could 
be overcome in the majority of cases. In the final survey, this scenario did 
not constitute a serious problem. In provinces where situations of this type 
were encountered, it was possible to find a solution by using more than 
one interviewee to obtain data concerning each family group, repeating the 
visit to the dwelling when necessary. 

Difficulties were occasionally experienced obtaining all of the data for 
some of the other residents of the dwelling where the interview was being 
developed. Nevertheless, in such cases it was usually possible to obtain suf- 
ficient data about the other residents to determine whether they were eligible 
for inclusion in the survey, and thus to proceed to the selection process. 


3.6 Final comments 


What is the future for undertakings like the ENI? The choice between gen- 
eral surveys or surveys focused on immigrants is always open to debate. 
General surveys are preferred when the main aim is to compare native and 
foreign-born populations, or when the main objective is to analyse the inte- 
gration of immigrants in work, education, housing, social life, political par- 
ticipation and other fields. International recommendations always empha- 
sise the importance of a deeper use of existing sources and a broader dis- 
semination of results among potential users (UN 1998; UNECE-Eurostat 
2008; Santo Tomas, Summers & Clemens 2009). Eurostat has also been 
working to allow researchers and public bodies to use many statistical 
sources virtually combined to produce aggregated information with added 
value (Eurostat 2007). 

However, in practice general surveys have many limitations when trying 
to obtain relevant information that might help to provide answers in politi- 
cal debates about immigration and international migration policies. 
Researchers, policymakers, citizens and social organisations perceive a 
deep gap between the desired objectives of immigration policies and em- 
pirical information about the flux of new immigrants, their relationship to 
established immigrant populations, returning emigrants and other emi- 
grants. The ENI provides a rich statistical portrait of the Spanish immigrant 
phenomena at its highest point (some 1 million entrances were recorded in 
2007). However, it is difficult to imagine the ENI as a cyclical undertaking, 
with its broad scope including all foreign-born residents of Spain. No repe- 
tition of the survey is foreseen at present. 
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Nevertheless, several lessons can be drawn from this experience for fu- 
ture research. First, the 2011 general population census and other related 
statistical work are important occasions to rethink usual survey procedures 
and to consider what additional information we need about the migrant 
population living in Spain. Second, focusing more on the medium term, 
some crucial aspects need to be explored related to the socio-economic 
characteristics of recent migrants and a new survey is being developed by 
INE to cover this topic (Resident Population and Recent Immigrants 
Survey). The final major lesson is the need to deepen the use of general 
surveys, to include more migrant-related variables, to monitor the quality 
of information collected and to enlarge the dissemination of data focused 
on immigrants. 


Appendix 
Estimators 


The main results of the surveys refer to dwellings and individuals. In both 
cases, the methodology used to obtain the estimation was the one tradition- 
ally used by INE in its household surveys, that is to say, ratio estimators 
with calibration techniques: 

1) Estimator based in the sample design (Horwitz-Thomson estimator). This 
is an unbiased estimator that takes the sample design into account. Weights 
were calculated as the inverse of the inclusion probability of the element. 

2) Non-response adjustment. A number of classes were considered in the 
adjustment on non-response, with the design weight inflated by the inverse 
of the response rate. These classes take into account provinces and size of 
dwellings. The size of a dwelling was calculated by considering the num- 
ber of foreigners residing in it belonging to the target population of the sur- 
vey. Dwellings with one, two and three or more persons were considered. 
3) Calibration techniques. The weight calculated following the previous 
two steps was re-weighted with the objective of adjusting the survey esti- 
mates to the information obtained from external sources. Calibration 
weights were calculated using the macro Calmar from the French National 
Institute of Statistics and Economic Studies (INSEE). The auxiliary varia- 
bles used in this survey were seven groups of nationalities, by 
Autonomous Communities, provided by INE’s population projections. 


Sampling error 


The survey provides sampling error estimations for the main characteris- 
tics. The procedure used to estimate the variance was the ‘Jackknife 
Method” applied in multistage sampling. This re-sampling technique 
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usually provides sufficiently accurate estimates of the variance of the esti- 
mator of a characteristic Y, V(Y). 
In the tables we publish the relative sampling error, or coefficient of var- 
iation, given by the following expression: 
v(Y) 


CV(¥) = — 


Sampling errors allow the construction of confidence intervals, in which 
the unknown value of the target characteristic is contained with a probabil- 
ity fixed in advance. For example, according to theory the following 
interval, 


(7 - 1,36 UD, X+1,96 ii). 


encompasses the true value of the parameter with a probability of 0.95. 


Notes 


1 Encuesta Nacional de Inmigrantes. 

2 The opinions expressed in this chapter are those of the authors and do not necessarily re- 
flect those of the INE. 

3 Many studies have tried to assess the extent to which these sources underestimated effec- 
tive flow; for an example see Babiano and Farré (2002). 

4 Colectivo IOE (1986); Muñoz and Izquierdo (1989). 

5 The effort involves a wide-ranging institutional partnership given the highly decentralised 
nature of Spanish government. 

6 Every registered resident must provide the following data that is included in the “Padrón”: 
name and surname, sex, permanent address, place and date of birth, Identity Card number 
(for foreigners, other documents that serve for this purpose). 

7 Grouped in the ‘Population and Society” Study Group (GEPS) based in the Complutense 
University in Madrid, under the leadership of David Reher, in collaboration with Luis 
Cortés Alcalá, Fernando González Quiñones, Miguel Requena Diez de Revenga, Alberto 
Sanz Gimeno, María Sánchez Domínguez and Mikolaj Stanek. 

8 The details of the tasks, commitments and obligations were concretised in an agreement 
between the INE, the Ministry of Labour and the Complutense University of Madrid, 
dated 16 October 2006. 

9 With the questionnaire that is currently used in the Family Budget Survey one cannot 
identify those who were born abroad, but it has been decided that it was better to intro- 
duce this aspect at a time it is deemed to be feasible (given that the size of this group is 
already considered relevant), rather than to generate a parallel effort in one of the system’s 
most complex and delicate statistical operations. 

10 Among the most important not yet mentioned is that it includes Spaniards born abroad 
who spend occasional periods in Spain. 

11 A description of the survey errors associated with this sampling design, as well as the es- 
timators used in the ENI, can be found in an appendix to this chapter. 

12 Since these interviewers had to cover interviews all over Spain, some of the (few) inter- 
views that required their intervention were done by phone. 
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4 Enhancing representativeness in highly dynamic 
settings: Lessons from the NEPIA survey 


Sebastian Rinken! 


4.1 Introduction 


This chapter is aimed at those migration researchers who live in an imper- 
fect world, methodologically speaking. Specifically, it addresses a series of 
challenges that arise when no sufficiently comprehensive sample frame is 
available, thus making the use of standard probability sampling either out- 
right impossible, or else unacceptable in terms of the share and characteris- 
tics of the target population covered by that procedure. 

Such was the situation with the NEPIA survey,” the fieldwork of which 
was conducted in the spring of 2003 in the Southern Spanish region of 
Andalusia. Located at the south-western edge of Europe, just across the 
Strait of Gibraltar from the African continent, Andalusia is home to about 
eight million people. Its vast shoreline attracts millions of foreign tourists 
each year. Starting in the 1980s, an increasing number of people from 
Northern and Central Europe came to settle more or less permanently in 
Andalusia, while the second half of the 1990s marked the onset of sizeable 
immigration flows from economically less developed countries. 

Aiming to gain solid empirical knowledge on the social situation of 
these latter migrants, the NEPIA study was commissioned in June 2002. 
The study was well-funded, with most of the project’s budget of close to 
half a million euros provided by the European Social Fund (ESF); indeed, 
the initiative for conducting large-scale social research on the living condi- 
tions of migrant workers in Andalusia had originated in Brussels, probably 
due to concerns sparked by the violent El Ejido incidents of February 
2000, which had made international headlines. However, the ESF’s gener- 
ous funding came with a tight and non-negotiable deadline (31 December 
2003) for the whole project to be completed. Apart from the survey on 
which this chapter focuses (see Pérez Yruela & Rinken 2005 for the broad- 
er picture), the NEPIA study comprised a series of additional tasks. Thus, 
from recruitment of a core research team of ten full-time staff to the onset 
of data collection, just five months (October 2002 through February 2003) 
were available for preparing the survey. Hence, the project’s success 
hinged not just on finding workable solutions to the various challenges that 
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will be discussed in the next section, but also on those solutions” smooth 
applicability. Any major misstep or delay would have derailed the whole 
project. 

Foremost among those challenges was the lack of a sufficiently compre- 
hensive and practically viable sample frame. As explained below, several 
flaws limited the usefulness of the population register (*Padrón”), which 
five years later served as sample frame for the ENI survey (chapter 3 in 
this volume). However, even if we had rated the “Padrón” a feasible sample 
frame, we could not have used it as such in the NEPIA study: probably 
due to a combination of party politics and European integrative goodwill, 
the NEPIA was explicitly required to describe Andalusia's immigrant pop- 
ulation regardless of any particular administrative condition, such as hold- 
ing a residence permit or signing up at the town hall as a local resident. 

This chapter’s structure is simple: after spelling out the main challenges 
faced by the research team (section 4.2) and explaining the corresponding 
decisions taken (section 4.3), both formal and indirect indicators are used 
to gauge the quality of data obtained (section 4.4). The conclusions (sec- 
tion 4.5) focus on assessing the risk-benefit equation of NEPIA-style 
procedures.* 


42 Major challenges 


The chapter title alludes to the highly dynamic nature of international mi- 
gration in Andalusia. Spain has gone from being a labour-exporting coun- 
try to a labour-importing one. From a comparative perspective, this demo- 
graphic shift is especially remarkable for its speed and magnitude: from 
2000 to 2005, the country attracted about 500,000 new international mi- 
grants per year (Rinken 2007). Among all OECD countries, the increase of 
foreign residents in Spain throughout the past decade has been second only 
to the United States in absolute terms, with Spain leading the table in per 
capita values (OECD 2008). For every ten Spaniards living in Spain in the 
late 1990s, there was in 2010 an eleventh inhabitant who was born abroad. 
To a somewhat lesser degree than in regions such as Madrid and 
Catalonia, and with some particular traits in terms of migrants’ origins and 
occupational profiles, the evolution in Andalusia has paralleled this general 
trend. Starting from about 100,000 a decade ago, the number of foreigners 
living in Andalusia had swelled to 670,000 by the end of 2008, according 
to the “Padrón” (OPAM 2009). 

While this dynamism was clearly discernible at the time of the planning 
of the NEPIA survey, there was no data source that would have enabled us 
to properly account for it. To express the same thing less euphemistically, 
there were no data that could have provided an adequate sample frame 
both in terms of inclusiveness and practical usefulness. The ‘Padrón’, a 
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unique database relying on the population registries of Spain’s municipal- 
ities, was, at the beginning of the decade, elaborated with astonishing de- 
lay: in the autumn of 2002, the latest available data referred to 1 January 
2000, putting the size of our target population (foreigners from economi- 
cally less developed countries living in Andalusia) at only 56,300. We 
knew this number was outdated, since at the end of 2001, persons from 
our target nationalities that held valid residence permits already amounted 
to 92,000, according to official records. This latter number clearly referred 
to a subgroup of the study population, since by all accounts, a significant 
share of immigrants in Spain lacked residence papers. 

But even if up-to-date ‘Padrón’ figures had been available, including the 
complete addresses needed for defining interviewer routes based on a clas- 
sic probability sample, we were sceptical as to their practical usefulness as 
a sample frame. The Spanish part of the NIDI study (Arango, Garcia- 
Pardo, Laseca & Martinez 2000) had tried to use “Padrón” addresses for ac- 
tually locating interviewees, with rather discouraging results. Even if we 
had been able to use updated data to define a probability sample and trust 
there was a high enough likelihood of locating a sufficient proportion of 
the selected subjects, the research objectives explicitly required us to side- 
step any administrative classification and describe the target population as 
a whole. The ‘Padrón’ establishes an surprisingly low bar for registering as 
an inhabitant (e.g. a bill showing both name and address will do); however, 
registration is still an administrative procedure that actually depends on the 
initiative of the individual to come forward. At the time of planning for the 
NEPIA survey, it was unclear to what extent this registry captured the for- 
eign population adequately, as people lacking residence papers were ex- 
pected to be far less likely to seek registration at the town hall than their 
‘authorised’ peers.* Adding to the problem of under-coverage, part of 
Andalusia’s immigrant population was known to suffer extremely poor liv- 
ing conditions. The above-mentioned El Ejido incidents had occurred in a 
greenhouse-dotted area where a sizeable share of migrants were deprived 
of anything resembling ‘normal’ housing and hence, were much less likely 
than other segments of the migrant population to register themselves in the 
‘Padrón’. 

In short, the NEPIA survey’s first and foremost challenge was to gener- 
ate a reasonably representative description of a population that could not 
be approached with the standard tools of survey methodology. Put bluntly, 
we were asked to square a circle. How would we select — or even encoun- 
ter — our interviewees without some population register from which to 
draw a probability sample? And how could we solve that problem without 
incurring a — potentially crippling — selection bias? 

A second and closely related challenge consisted of adequately address- 
ing the target population’s diversity in dimensions such as national origin, 
geographic location, occupational profile, living conditions, administrative 
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status and linguistic skills. In terms of origin, according to available data, 
between 35 and 40 per cent of the target population was from Northern 
Africa (primarily Morocco) and roughly the same share was from Latin 
America (with considerable numbers of Argentinian, Ecuadorian and 
Columbian nationality, respectively). Immigrants from the sub-Saharan 
part of the African continent were far less important numerically, but 
highly relevant symbolically, due to those infamous, perilously inapt and 
notoriously crowded boats ferrying would-be immigrants across the Strait 
of Gibraltar (in more recent years, much of that flow has been heading 
toward the Canary Islands instead, due to improved border control in the 
Strait). Also present, albeit numerically far less significant than Latin 
Americans and North Africans, were nationals of East European and 
Asian countries. The dispersion of the target population’s origins implied 
the dilemma of either focusing on specific groups (defined by geopolitical 
area or even nationality), thus substantially easing the complexity of field- 
work, or else accepting the added difficulties of a broader approach, such 
as obtaining enough observations for chosen sub-categories to be statisti- 
cally significant and managing a potentially Babylonian situation in terms 
of linguistic diversity. 

A similar dilemma arose regarding the location of immigrants within 
Andalusia. The pressing concern of ensuring viable fieldwork logistics, 
which may have propelled a geographical focus on selected areas with a 
relatively high volume of immigrants, had to be balanced against the objec- 
tive of representing as well as possible the occupational and social diver- 
sity of the target population. This regards partly the size of the municipal- 
ities, and partly their socio-economic profile. Arguably, which sorts of oc- 
cupations are accessible to migrants depends to a great extent on the 
opportunity structure of the labour market, which in turn depends on the 
sectors of activity that dominate the local economy. As to living conditions 
and administrative status, these are also partly related to the general socio- 
economic setting at the place of residence. 

Our decisions regarding 1) the general survey framework and 2) the in- 
tent of maximising social and cultural diversity within the target population 
implied that 3) fieldwork management — and specifically, quality control — 
was yet another major challenge to be concerned about from the start. The 
following section seeks to explain how the NEPIA addressed these three 
interrelated challenges. We will hence recapitulate the reasons that induced 
the research team to prefer one particular course of action over its possible 
alternatives, deciding on the basis of imperfect clues regarding the compa- 
rative risk profile of those options and, as indicated above, under the pres- 
sure of a tight deadline. 
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4.3 Key decisions 


The research team's decisions can best be discussed in three steps that 
match the three main challenges sketched above, though most of the deci- 
sions relate to various dimensions. 


General sampling approach 


With regard to the plainly crucial challenge of generating a valid sampling 
approach, our first decision was to recognise the impossibility of obtaining 
a sufficiently complete list of the members of the target population from 
which to draw, with equal likelihood of selection, fully identified units — 
individuals or households — that could then be contacted for data gathering. 
This decision was obviously triggered by the context conditions at that 
time; in the concluding section, we will address the hypothetical question 
of how we would decide on that point today. Secondly, short of a full prob- 
ability approach, we figured that more up-to-date “Padrón” data than those 
already published would provide a reasonably reliable tool for estimating 
the general parameters of the study universe, such as origin (geopolitical 
groups of nationalities), gender (proportions within each geopolitical 
group) and location (municipality). In other words, we decided to use the 
“Padrón” for everything that is usually done with the source data (frame) 
from which to draw a sample, except for the last step of exactly defining 
the unit of data collection. 

For us, the “Padrón” provided a proxy universe from which a proxy sam- 
ple was drawn. Not only were we well aware that there was no full match 
between the individuals registered in the “Padrón” and the target population 
(for starters, the latter also comprised people who had not come forward to 
register as inhabitants at their town hall); we also accepted that the ‘Padrón’ 
data would only go half the distance towards identifying a particular inter- 
viewee. To be specific, “half the distance” was a search profile containing 
the interviewee’s municipality of residence, geopolitical region of origin and 
gender as compulsory items and his or her specific nationality as a desirable 
one that, according to circumstances in the field, could be substituted by an- 
other nationality from that same geopolitical group. Inscription in the 
‘Padrón’ was not a prerequisite for interviewee selection, but a questionnaire 
item instead. Hence, in NEPIA’s approach to sampling, the only assumption 
made with regard to the immigrant population’s administrative status was 
that immigrants lacking “Padrón” registration would not be geographically 
distributed in ways systematically different from the settling patterns of im- 
migrants that did compute in the population register. We will come back to 
NEPIA’s interviewee recruitment procedures in the next section. 

To obtain our proxy-universe of up-to-date “Padrón” data, the NEPIA 
team collected such data directly from those municipalities that accounted 
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for 90 per cent of the registered immigrant population in each of 
Andalusia’s eight provinces. That share was measured by the latest pub- 
licly available ‘Padrón’. Focusing on localities with a significant presence 
of immigrants made this particular data-collection effort manageable. 
Moreover, including places with no or hardly any foreigners would have 
been pointless. Of the 170 municipalities that were approached, 150 coop- 
erated by providing recent and disaggregated (gender, nationality) data. For 
the remaining non-contacted 620 municipalities, the latest published data 
were used. As a result, NEPIA’s proxy universe comprised about 148,500 
foreigners from economically less developed countries. This figure tripled 
the corresponding number in the last publicly available data. Rather than 
the absolute number, what mattered for the NEPIA research process was 
the possibility of plausibly estimating the proportions (place of residence, 
country of origin, gender) that were relevant for drawing our sample of in- 
terviewee search profiles. The fieldwork was to be oriented by but not re- 
stricted to people enrolled in the population register. 


Maximising diversity while minimising selection bias 


As mentioned earlier, the study’s target population did not comprise for- 
eigners from any specific country of origin, but only immigrants from eco- 
nomically less developed countries. This criterion excluded people from 
countries belonging at that point to the European Union or other highly in- 
dustrialised nations, but it included a vast range of countries, the propor- 
tional weight of which ranged from about one third (Moroccans) to almost 
zero. When combining all the eligible countries of origin into broader 
groups, Asians and sub-Saharan Africans accounted for just 5-6 per cent 
each, East Europeans for twice that figure, and immigrants from Latin 
American and North African countries close to 40 per cent each. To meet 
the goal of describing the immigrant population, we could have opted for a 
proportionate sample with regard to the variable ‘country of origin”; how- 
ever, this would have made comparisons between distinct groups of mi- 
grants largely impossible. Hence, the sample was stratified, with equal 
numbers of interviews to be conducted with individuals from each of the 
following five geopolitical regions: Northern Africa (including Middle 
Eastern countries that border the Mediterranean), sub-Saharan Africa, Latin 
America, Asia (excluding Japan and the Middle East), and Eastern Europe. 

To account for the diversity of opportunity structures across Andalusia’s 
vast territory, the sample was also stratified with regard to the economic 
profile of the interviewee’s place of residence. To this effect, Andalusia’s 
770 municipalities were classified into four categories: tourism-driven 
coastline, intensive agriculture, urban area, and a residual group that 
mainly comprises the region’s rural interior. The size of the total sample 
(N=1,800) balanced viability constraints (deadline) and budget 
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considerations against the need to obtain statistically significant sub-sam- 
ples. The fixed quotas for region of origin and area of residence (N=360 
and N=450, respectively) were combined with proportional quotas for gen- 
der. To obtain adequate proportions across subcategories, results were 
weighted according to the “Padrón” data as of autumn 2002. 

The sample’s cross-stratification (Table 4.1 shows the real sample ob- 
tained; N=1,797) constituted a first step toward reflecting the target popula- 
tion’s internal diversity. According to their specific typology, interviewee 
search profiles were then assigned to eligible municipalities, with a mini- 
mum number of eight interviews per locality to prevent an excessive geo- 
graphical dispersion of fieldwork. Municipalities whose volume of immi- 
grant residents was sufficient for their share to exceed eight interviews 
were automatically selected for fieldwork, whereas among the rest, field- 
work locations were randomly selected, the selection probability of any 
given municipality being proportional to its share of immigrant residents. 
Again, all these calculations were made on the basis of the “Padrón” figures 
collected by the NEPIA team. 

As mentioned earlier, ours was not a fully operative sample, complete 
with addresses or even names. The NEPIA sample specifies each inter- 
viewee’s gender, municipality of residence and geopolitical region of ori- 
gin, plus an indication of the preferable country of birth. For the interview 
to come about, the interviewer had to make contact with an individual 
matching each particular search profile. This last step is obviously crucial 
for data quality: the general philosophy of survey methodology would de- 
mand an equal selection likelihood of any individual that matches a given 
profile, a requirement that is usually met by random selection from a 


Table 4.1 Obtained NEPIA sample (March-April 2003) 


Socio-economic profile of municipality 


Total Large Touristic Intensive Rural 

cities coastline agriculture interior 
Geopolitical region All M W M W MW M W M W 
of origin 
Asia 358 203 155 70 53 77 67 4 5 52 30 
Eastern Europe 359 165 194 22 30 39 45 58 62 46 57 
Latin America 360 160 200 44 58 47 57 19 22 50 63 
Northern Africa 361 239 122 47 27 45 23 82 37 65 35 
Sub-Saharan Africa 359 277 82 88 24 28 8 122 3 39 11 
Total 1,797 1,044 753 271 192 236 200 285 165 252 196 

463 436 450 448 


Source: NEPIA survey 

Note: The sample is cross-stratified by socio-economic profile of fieldwork municipality and 
geopolitical region of interviewees” country of birth; proportional quotas were used for inter- 
viewees’ gender, abbreviated here as M and W. 
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reasonably inclusive sample frame with sufficiently operative contact infor- 
mation. As explained earlier, this precondition could not be met here. 

NEPIA’s solution to this conundrum was to diversify the contact points 
for recruiting interviewees. On the assumption that migrant self-help and 
support associations were both highly prone to create selection bias (sup- 
port is arguably sought by the needy only) and likely to be favoured by in- 
terviewers (especially those dispatched to an unfamiliar municipality), this 
means of making contact with interviewees was restricted to a maximum 
of 20 per cent of each interviewer’s workload. On similar grounds, snow- 
balling was explicitly forbidden. To improve the probability of adequately 
representing the breadth of social situations existing within each of the var- 
ious search profiles, each interviewer had to recruit at least 80 per cent of 
their interviewees at contact points belonging to a variable mix (between 
20 and 50 per cent each) of the following three categories: 1) employment, 
2) residence and 3) leisure. Examples for employment-related recruitment 
points were middle- and upper-class residential neighbourhoods (for inter- 
viewing domestic service personnel), restaurants and construction sites, al- 
ways provided that the interviewee was employed there. ‘Residential’ re- 
cruitment spots included any public place and facility (e.g. schools), as 
well as any sort of housing unit used by immigrants. Leisure-related sites 
included bars, parks, sports facilities and centres of worship. 

When establishing this procedure, we were not aware of Blangiardo’s 
(1996, 2000) work on centre sampling. Our approach is similar in that con- 
tact with interviewees was established at a variety of recruitment points. 
However, whereas Blangiardo sought to approach equal-selection-probabil- 
ity of all the individuals frequenting a given set of contact points, NEPIA 
placed the primary emphasis on the sampling of search profiles. Later, at 
the stage of actual fieldwork, an effort was made to diversify the contact 
points where individuals were recruited to match those profiles. 
Blangiardo’s procedure implies the interviewing of any individual that can 
be located at one of various contact points, whereas NEPIA’s recruitment 
point constitutes a tool for adequately implementing a sample of specific 
search profiles, drawn from the proxy-universe of ‘Padron’ data (but with- 
out making ‘Padrón’ inscription an interviewee recruitment requirement). 
While for NEPIA, each interviewee’s gender and country of origin (or 
broad geopolitical region of origin) were defined a priori on the basis of a 
proxy-universe, the variables ‘gender’ and ‘nationality’ are part of the esti- 
mates generated by Blangiardo’s method. 


Data collection and quality control 


The decisions taken with regard to the sample’s internal diversity implied a 
complex data-collection process. In terms of social diversity, our goal was 
to capture any type of situation, ranging from the homeless to well- 
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established professionals and entrepreneurs. In terms of cultural and lin- 
guistic diversity, the establishment of fixed quotas for proportionally less 
relevant groups of origin would have been pointless without measures 
aimed at overcoming the language barrier. By all accounts, migrants who 
are not fluent in the host country’s language are likely to differ substan- 
tially from the rest on key variables such as length of stay, administrative 
status, employment and residential situation. 

Questionnaire design aimed to take account of these added challenges. 
To accommodate social diversity, specific question-and-answer paths were 
designed for different categories of people. For example, respondents who 
had reported precarious housing conditions (tents, abandoned cars) would 
not be asked to report on equipment such as central heating. In short, one 
goal when designing the questionnaire was to avoid any query that would 
annoy the respondent as superfluous or disrespectful, considering the infor- 
mation obtained previously. Also, a relatively large portion of the question- 
naire permitted open-text responses to be coded later on, thus helping to 
prevent non-response or discontinuity. While being markedly interviewee- 
friendly in various ways, the questionnaire included many requests for sen- 
sitive information, such as the means of transportation used for entering 
the country, current administrative status, and status regarding social secur- 
ity. The questionnaire was translated into four additional languages (Arab, 
Russian, French and English), apart from the standard version in Spanish. 
Since fieldwork was to be carried out during the months of March and 
April, February 2003 was the reference period for many questions. When 
applicable, standard classifications were used to obtain data that would be 
comparable with sources such as the general labour force survey. 

Staffing of fieldwork teams entailed a comparative risk-benefit assess- 
ment of professionalism versus proximity. The IESA regularly runs data- 
collection operations across the region. NEPIA could have used that net- 
work of interviewers and supervisors, perhaps saving the project team sig- 
nificant time and trouble. Nevertheless, we decided to forego that option 
and build a specific data-collection operation instead, primarily due to con- 
cerns about trust and cooperation on the interviewees’ part and secondarily, 
to enhance proficiency in languages other than Spanish. The initial project 
team comprised four full-time staff members (including the general project 
coordinator) at headquarters, plus seven full-time provincial coordinators. 
The latter took on the main responsibility for supervising teams of around 
ten interviewers, recruited preferably to obtain the cultural affinity and lan- 
guage skills relevant for interacting with the predominant groups of origin 
in each province’s share of the sample. Training of fieldwork teams was 
provided by headquarters staff. A detailed interviewer manual was drafted, 
with instructions regarding both general conduct and the NEPIA question- 
naire specifically. The payment scheme also had to be generated afresh, 
since the absence of pre-established door-to-door routes meant that there 


94 SEBASTIAN RINKEN 


was little indication of the amount of time needed for a given interviewee. 
Our take on this was to establish pre-set rates that varied according to the 
distance of fieldwork locations from interviewers” homes, applicable to 
successfully conducted interviews only. Payment was generous, to motivate 
interviewers to meet their assigned commitments. 

Especially strict quality control was, again, a necessity derived from ear- 
lier decisions. Ceteris paribus, interviewer misconduct (outright cheating, 
interviewer bias, etc.) can be assumed to be more prevalent in a relatively 
inexperienced ad hoc fieldwork team that has no sense of continuity in this 
line of activity, once the particular one-off study has been concluded. To 
make the threat of dismissal somewhat more effective as a preventive 
measure, NEPIA’s approach combined a large carrot (generous payment) 
with an equally large stick, which came in the way of three-step quality 
control on the part of the corresponding province’s coordinator, central 
project staff and IESA coding professionals, respectively. In the first two 
steps, each interviewer’s entire submitted workload was revised for repeti- 
tive, incoherent or otherwise strange answer patterns, including information 
on interviewee recruitment points (which had to be specified exactly); the 
questionnaire included some ‘red flag’ items that were specifically checked 
as telltales for possible cheating. Additionally, the last step entailed phone 
calls to a large share of respondents. To make this possible, interviewers 
were asked to obtain operative contact information, such as a mobile phone 
number. 

In a general political context marked largely by the migration-control 
discourse of the Spanish national government (run by the conservative 
Partido Popular at that time, in contrast to the Socialist government of 
the Andalusia region), mistrust toward interviewers was a distinct possi- 
bility, especially on the part of undocumented migrants. Also, it was un- 
clear whether the interviewers’ institutional affiliation to the IESA and 
the concept of confidentiality would be understood (and taken for real) 
by all interviewees. Despite these circumstances, which implied that the 
requirement of obtaining contact information might have endangered the 
timely completion of fieldwork, we made it clear that a low percentage 
of questionnaires including such information would raise suspicions of 
possible interviewer misconduct, thus triggering especially intense 
screening. 


4-4 Outcome measures 


To gauge the quality of the data generated by the NEPIA survey, we will 
first discuss some formal indicators inherent in the data-collection effort. 
Subsequently, substantive results will be compared with data produced by 
other sources. 
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Formal quality indicators 


The procedures employed by NEPIA entail a hitch of considerable magni- 
tude: data accuracy cannot be measured as usual. Random error can be 
computed technically for the total sample (+/- 3.15 per cent at a confidence 
interval of 95 per cent) and for each sub-sample (+/- 5.2 per cent for each 
category of origin and 4.6 per cent for those of location), but since the pre- 
requisite of a probability sample is not fully applicable here, those calcula- 
tions provide only tentative indications. To be sure, even if NEPIA had 
been carried out with standard survey methodology, the relatively small 
sample size and the decision to cross-stratify that sample generated ranges 
of random variance that are anything but negligible. Hence, this survey — 
like most quantitative data-collection operations, except extraordinary large 
ones — provides a tool essentially for assessing approximate proportions, 
rather than subtle differences. In this sense, in NEPIA, as in any moderate 
and small-size survey, there is a fictional element to the presentation of ex- 
tremely detailed numerical results, complete with decimals. This restriction 
regarding data exactitude should be perfectly admissible for a survey of the 
characteristics we refer to here. 

Now, in the case of NEPIA, the likelihood of selection is not necessarily 
equal for all members of the corresponding category of people (defined by 
municipality of residency, gender, and country/region of origin) in the deci- 
sive last step of interviewee recruitment, as it was not purely randomly 
driven. Since initial contact with potential respondents was essentially 
made in public places of various sorts, anybody shumning these was a pri- 
ori less likely to be recruited for the research interview. Generally speak- 
ing, a migrant with residence papers and a job in the formal economy 
would be more likely to be interviewed while shopping in a supermarket, 
for example, than a migrant lacking both. Similarly, a migrant living in a 
shack at the periphery of a given fieldwork location is a priori less likely 
to be interviewed than somebody living and working close to that locality’s 
main traffic nodes. 

The research team took measures to minimise that source of systemic er- 
ror, for example, by devising a list of interviewee recruitment points which 
included sites where the selection of individuals in socially and legally pre- 
carious situations was likely to occur. About 8 per cent of interviewees 
were contacted via self-help or charitable organisations. The remainder was 
distributed fairly evenly among leisure, residential and employment-related 
contact points, with between 28 and 36 per cent each. Arguably, the nature 
and dimension of potential visibility bias varies according to the size and 
nature of fieldwork locations. For example, in Andalusia’s agricultural 
areas, often undocumented seasonal workers are forced into visibility, gath- 
ering at public places to procure short-term employment. Hence, it is worth 
pointing out that our fieldwork was conducted in a total of 108 
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municipalities, ranging from major cities and towns with sizeable immi- 
grant communities to small municipalities with relatively few foreign resi- 
dents. Also, 70 per cent of valid interviews were carried out with the first 
eligible person approached. However, as stated, the degree of data accuracy 
obtained cannot be fully captured with intrinsic measurements. The section 
on ‘substantive quality indicators’ is thus of crucial interest, referring as it 
does to tools of external validation. 

Such validation is warranted with regard to a second potential source of 
error as well. As described earlier, the NEPIA team put tremendous effort 
into quality control throughout all the stages of fieldwork, but we cannot 
exclude that the survey results may be subject to some interviewer bias. 
The two main forms of interviewer bias that we encountered were outright 
cheating, on one hand, and possibly misleading ad hoc translations of the 
questionnaire, on the other. With regard to the former, our quality control 
procedures detected some cases of complete disregard of mandatory field- 
work procedures, such as ‘agenda scrolling’, that is, rather than going out 
and seeking respondents, friends and acquaintances were interviewed. The 
dismissal of unprincipled staff and the dropout of others led to a substantial 
reduction of interviewer teams throughout the fieldwork period; typically, 
the last few dozen interviews in each province were conducted by a resid- 
ual group of especially motivated and reliable interviewers. As to the sec- 
ond aspect, the multilinguistic nature of data collection in the field led to 
an unanticipated number of ad hoc translations of the original Spanish 
questionnaire. Surprisingly little use was made of the foreign-language 
questionnaires (in about 2 per cent of the 1,797 valid interviews), whereas 
about one fifth of the total were Spanish questionnaires administered with 
the help of at least some ad hoc translating on the part of the interviewer. 
About two thirds of those cases involved translations or clarifications in 
the Arab language. Optimists would deduce that our goal of linguistic in- 
clusiveness was indeed met. Pessimists, however, would note that the un- 
controlled nature of those spontaneous translations raises the possibility of 
inaccuracies and outright errors, some of which may have gone undetected 
in subsequent screening due to the limited language skills of quality-con- 
trol staff. 

That said, it is noteworthy that 70 per cent of interviewees provided a 
contact telephone number for quality control. This figure can be interpreted 
as an encouraging sign regarding data quality, especially if compared with 
the mobile telephone penetration rate of about 80 per cent in our sample. 
From the interviewees’ perspective, the readiness to provide a contact tele- 
phone suggests that the interview did not generate distrust, which in turn 
might be associated with truthfulness on the interviewees’ part. From the 
interviewers’ perspective, the high proportion of questionnaires with con- 
tact information implies that our quality-control procedures were taken 
seriously. As to perceived quality, most interviews were rated as ‘good’ or 
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“very good” by the interviewers, with relatively low marks for interviews 
with Asians and especially high marks for Latin American respondents 
(82.3 and 95.5 per cent, respectively, of these latter were rated as “good” 
and ‘very good’). Sincerity ratings were similar, with 96 per cent of Latin 
American respondents receiving ‘good’ or ‘very good’ marks, as compared 
to about 85 per cent of Asian and North African interviewees. 

The low percentage of non-response regarding sensitive information is 
also a positive sign. For example, only about | per cent of interviewees de- 
clined to indicate their means of transport for reaching Spain, while about 
a quarter of male interviewees from sub-Saharan Africa and one fifth of 
male interviewees from Northern Africa reported having crossed the Strait 
of Gibraltar illegally in a patera” (figures are far lower for female respond- 
ents). At about 8.5 per cent in the weighted sample as a whole, non-re- 
sponse regarding interviewees’ income was in line with comparable items 
in other datasets. 


Indicators of substantive data quality® 


In the NEPIA survey, the question regarding access to mobile phones was 
part of a battery of items regarding facilities in the respondent’s housing 
unit. Since that battery also included items such as air conditioning and 
central heating, it was omitted from the question path for interviewees who 
had previously stated that they were living in highly precarious conditions, 
such as abandoned cars and the like. Such situations were reported by 
about 6 per cent of the weighted NEPIA sample, whereas about 8.5 per 
cent reported to be living in a single room (hotel, sublet, in-house domestic 
service, etc.) and close to 85 per cent said they were living in a full-size 
flat or house; 13 per cent of the latter said they owned the house and 78 
per cent rented; 7 per cent or so lived in housing units made available by 
employers or kin. 

This example (for more detailed data regarding housing, see Rinken & 
Herron 2004) illustrates that the NEPIA survey was successful in including 
persons living in precarious conditions in the data-gathering operation. 
This capacity differentiates NEPIA from surveys that convert an adminis- 
trative register regarding relatively ‘normal’ living conditions into the sam- 
ple frame. For instance, the ENI, carried out in 2007 by the National 
Statistics Institute (see chapter 3 of this book) basically excludes most sorts 
of crudely substandard housing from the outset, relying as it does on ad- 
dresses contained in the population register. Hence, the above-mentioned 
drawbacks of NEPIA’s approach have to be pondered in association with 
the pitfalls of more classic survey methodology. 

Under-coverage stands out as the foremost hitch in applying (relatively) 
standard survey methodology to a highly dynamic setting, such as a popu- 
lation of international migrants whose relationship with the receiving 
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society’s institutions and administrative procedures is, precisely, part of the 
empirical situation to be clarified. Within the boundaries of a given frame 
for “pure” probability sampling, both systemic and random error can be 
controlled for and measured much more adequately than by generating a 
proxy-sample of search profiles, as was the case with NEPIA. However, 
when implemented properly, such search profiles generate potentially very 
inclusive results in terms of the target population’s diversity of social and 
administrative situations. In contrast, the boundaries of a classic sample 
frame will categorically exclude part of the theoretical target population. 

To give an idea of the magnitude of such under-coverage, one quarter of 
NEPIA’s respondents indicated they had not registered in their local town 
hall’s “Padrón”. The social and administrative situation of this segment of 
NEPIA’s target population differs substantially from the rest, with an al- 
most tenfold increase of manifestly precarious residential situations (about 
17 per cent versus 2 per cent, respectively) and twice as high a proportion 
of individuals lacking valid residence papers (70 versus 35 per cent), to 
mention just two remarkable features. Since “Padrón” registration, adminis- 
trative status, and housing conditions pertain to the vast group of variables 
that in turn depend to a large extent on the respondent's length of stay in 
the host country, these differential distributions make perfect sense. From a 
methodological viewpoint, the data suggest that under-coverage is poten- 
tially a more serious source of error than the two sorts of bias (interviewee 
visibility and interviewer error) that may have reduced the accuracy of 
NEPIA results. NEPIA’s procedures were open toward categories of inter- 
viewees that are systematically precluded from participation by standard 
probability sampling, due to the latter’s dependence on administrative 
registers that almost inevitably miss out on the fringes of the theoretical 
universe. To put this argument very cautiously, the co-existence of home- 
less and home-owning individuals in the NEPIA sample can hardly be in- 
terpreted as evidence of systemic distortion. 

To what extent, then, can we trust that NEPIA’s findings reflect, with a 
reasonable margin of error, the empirical proportions in the target popula- 
tion? In the following, we will attempt to gauge the reliability and validity 
of NEPIA’s results by comparing them with other available data sources. 
The methodologies used for generating these various data imply that we 
should not expect full coincidence with the NEPIA results. Rather, the 
comparison aims at making sense of the data obtained by relating them to 
the methodologies applied in each case.’ We will first compare some basic 
socio-demographic traits from the NEPIA sample with the official ‘Padrón’ 
statistics. Second, NEPIA’s results regarding the share of immigrants with 
valid residence permits and social security enrolment, respectively, will be 
compared with official data concerning these aspects. Third and last, we 
will turn to the Andalusian sub-sample of the general Spanish Labour 


ENHANCING REPRESENTATIVENESS IN HIGHLY DYNAMIC SETTINGS 99 


Force Survey to cross-check some basic traits of the migrants” reported em- 
ployment situation. 

As mentioned earlier, the NEPIA study used largely self-collected 
“Padrón” figures to estimate the relative size of each sub-sample and organ- 
ise fieldwork across Andalusia’s vast territory. Those figures referring to 
all age groups were extracted from the respective town halls’ records in the 
autumn of 2002, except for a small percentage of entries that were re- 
trieved from the latest official “Padrón” (as of 1 January 2000) available at 
that time. The sample generated with the help of *Padrón”-based search 
profiles included any interviewee matching the defining traits of gender 
and geopolitical area of birthplace, regardless of whether that person had 
registered in the “Padrón”. Hence, the correspondence between the basic 
socio-demographic traits of the NEPIA sample (A) and the official 
“Padrón” as of 1 January 2003 regarding nationals of the countries that 
compose NEPIA's five geopolitical areas of origin (B) cannot be expected 
to be near perfect. 

However, as Table 4.2 illustrates, regarding aspects that were part of 
NEPIA’s obligatory search profile (geopolitical region of origin and 


Table 4.2 Socio-demographic traits of the NEPIA sample (A) and of corresponding 
nationalities according to official ‘Padrón’ statistics (B) 


Total Northern Latin Eastern Sub-Saharan Asia 
Africa America Europe Africa 

(A) (B (A) B (A) B (A) ( (A B A (B 
Relative weight of — = 39 35 37% 38% 13 15 6* 6* 5* 6* 
geopolitical region 
(% of total) 
Men 56% 56* 67* 69% 45% 44% 47% 49% 78% 76* 57%  60* 
(% of total) 
Under 30 43 405 42* 42* 41% 39% 57 44 50 38 39 32 
years of age 
(% of total) 
Predominant Morocco Morocco Argentina Romania Senegal China 
countries (column (32%) (31.5%) (83) (90) (30) (24) (41) (30) (34) (28) (51) (41) 
share according to Argentina Algeria Ecuador Russia Nigeria Pakistan 
A&B) (11) (9) 00 G) (27%) e 08) (05%) (19) (26) (059) 08% 

Ecuador Colombia Ukraine Guinea Bissau Philippines 


(10%) (10%) (26%) (25%) (16%) (19%) (16) (9) (10) (14) 


Sources: NEPIA survey (March-April 2003) and National Statistics Institute, INE (‘Padrén’, 1 
January 2003) 

Note: Columns (A) refer to the birthplace, columns (B) refer to nationals of the countries of 
origin included in the NEPIA target population. Columns (A) comprise individuals aged 16 
years or older; columns (B) refer to individuals aged 15 years or older. Values (A) and (B) 
that do not differ significantly from one another are marked with an asterisk. All figures dis- 
played are rounded. 
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gender), the sample proportions are generally very much in line with those 
of the official population statistics relative to the closest possible date. This 
is remarkable, since the statistics in question became available only when 
NEPIA’s results were already published. With respect to traits such as 
country of origin (which was a recommended but optional part of the 
search profiles) and age, the differences widen in some subcategories, but 
sample totals maintain a remarkable degree of similarity with the corre- 
sponding values from the population register. With regard to the higher 
proportion of young NEPIA interviewees, in comparison with ‘Padrón’ fig- 
ures, it is worth mentioning the sample's internal difference between 
“Padrón”-registered and unregistered interviewees, with shares of the 16-29 
age group being 39 per cent and 57 per cent, respectively. The first of 
those two values is almost identical to the official figure. As to the share 
of specific countries of origin, the discrepancies between columns (A) and 
(B) are in some cases likely to be due to accelerating demographic growth 
rates (example, for Romania), which translate into relatively large propor- 
tions of newcomers who were not (yet) registered in the “Padrón”. Still, in 
at least one case, the difference is potentially related to NEPIA selection 
bias (in the Asian subgroup, Chinese interviewees might be easier to re- 
cruit than other nationalities). 

As mentioned earlier, NEPIA managed to collect information on notori- 
ously sensitive aspects of immigrant life, including administrative status 
and social security enrolment. Table 4.3 compares NEPIA results in these 
regards with official statistics. Columns (B) refer to the numerical relation- 
ship between the values of the “Padrón” population registry as of 1 January 
2003, on one hand, and valid residence permits as of year’s end 2002 and 


Table 4.3 Administrative profile of NEPIA sample (A) and comparison of available 
official statistics regarding corresponding nationalities’ administrative 
status (B) (per cent share of total) 


Total Asia Northern Sub-Saharan Latin Eastern 

Africa Africa America Europe 

(A) (B) (A) (B) (A) (B) (A) (B) (A) (B) (A) (B) 

Valid residence 52% 54* 81 74 66 78 59 85 39 32 32* 33% 
permit 

Social security 37 45 71 62 41 57 45 57 31* 30% 26 44 
affiliates 


Sources: NEPIA survey (March-April 2003) and Ministry of the Interior (OPI 2003) 

Note: Columns (A) refer to a combination of birthplace and nationality (naturalised immi- 
grants are excluded), whereas columns (B) simply refer to nationals of the countries in- 
cluded in the NEPIA target population. Columns (A) comprise individuals aged 16 years or 
older; columns (B) refer to individuals of any age for data on residence permits and individ- 
uals aged 15 or older, for data on social security. Values (A) and (B) that do not differ signif- 
icantly from one another are marked with an asterisk. All figures displayed are rounded. 
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social security enrolment as of mid-January 2003, respectively, on the oth- 
er. These data do not procure an exact quantification of administrative ir- 
regularity, among other reasons because naturalisations are not considered 
here for lack of sufficiently disaggregated official data. Also, the share of 
social security affiliations is calculated in relation to the total population, 
not just the employed. That said, with a naturalisation rate of less than 5 
per cent and an employment rate of about 70 per cent (NEPIA data), the 
figures in Table 4.3 suggest a high level of irregularity with regard to both 
indicators, thus adding to existing empirical knowledge regarding the so- 
called Southern European model of managing migration (see Cebolla & 
González-Ferrer 2008; Arango 2000; Cornelius 1995). As to the compari- 
son between columns (A) and (B), the former can be expected to contain 
higher estimates of irregularity, since NEPIA data include people not regis- 
tered in the “Padrón”. 

This expectation is confirmed in the case of social security enrolment: 
the sample total and subtotals for three of the five groups of origin are sig- 
nificantly lower than the figure obtained using administrative data (column 
B). As for the one case that contradicts this trend, NEPIA data on the so- 
cial security enrolment of Asian migrants may be partly due to visibility 
bias, such as locating interviewees primarily in commercial establishments. 
Regarding NEPIA data on residence permits, an analogous remark might 
be due: the Asian sub-sample is again one of only two to show higher pro- 
portions of valid residence papers than the available administrative statis- 
tics. That said, in general, the degree of coincidence between the NEPIA 
results and available administrative sources is remarkable. Both means of 
measurement identify Latin Americans and Eastern Europeans as two 
groups of origin with astonishingly low proportions (about one third) of 
regular administrative status. This finding is consistent with the fact that 
immigration flows from Latin America and Eastern Europe accelerated at 
considerable rates throughout the years preceding the NEPIA survey, with 
new arrivals generally lacking the corresponding administrative requisites. 

The possibility of comparing the NEPIA results with data produced by 
the Spanish National Statistics Institute’s general Labour Force Survey 
(EPA)? is limited. At the relevant time (Ist quarter of 2003), a small share 
of that survey’s huge data-collection operation was directed at immigrants 
from economically less developed countries and residing in Andalusia; 
hence, statistically significant data are lacking for many subcategories. To 
cross-check NEPIA results, we will look at the employment situation 
(Table 4.4), sector of economic activity (Table 4.5) and the employment 
qualification level (Table 4.6). 

Regarding the employment situation, the coincidence between NEPIA’s 
results and data procured by the EPA is almost perfect concerning the total 
target population and some sub-samples (Latin Americans, Eastern 
Europeans), even though some of these values narrowly miss our threshold 
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Table 4.4 Employment situations of employed interviewees, NEPIA sample (A) and 
corresponding nationalities in Andalusian subsample of EPA (B), by 
subgroup of origin and total (per cent) 


Total Latin Eastern Northern Sub-Saharan Asia 
America Europe Africa Africa 


(A) (B) (A) (B) (A) (B) (4) (B) (A) (B) (A (B) 
Entrepreneur / 17 14 15% 12* 2* 4* 21 (9) 18 (29) 34 0 
autonomous 
professional 


Employee 79 82 83% 80* 94 96 71% (71)* 79 (71) 61 100 
Member of 1 2 1 4 0% O“ 2 (0) 1% (0)* 7 0 
cooperative 

Helping family 2* 2* 0 4 3 0 4 (0) 1* (0)* 2 0 
member 

Other 1 = 2 = 0 = 2 = 2 == 1 = 


Sources: NEPIA survey (March-April 2003) and INE (EPA Q1/2003) 

Note: In NEPIA, the reference month for employment-related questions was February 2003. 
Regarding origin, available EPA categories refer to Latin America, Eastern Europe, the whole 
of the African continent and a residual ‘Asia plus Oceania’; since the latter is numerically ir- 
relevant here, the NEPIA category 'Asia' has been maintained. EPA data with doubtful statis- 
tical representativeness are shown in italics and EPA data regarding African respondents 
are in parentheses. Values (A) and (B) that do not differ significantly from one another are 
marked with an asterisk. All figures displayed are rounded. 


Table 4.5 Sectors of economic activity among employed interviewees, NEPIA 
sample (A) and corresponding nationalities in Andalusian subsample of 
EPA (B), by subgroup of origin and total (per cent) 


Total Latin Eastern Northern Sub-Saharan Asia 
America Europe Africa Africa 


(A) (B) (A) (B) (A) (B) (A) (8) A (B (A (B 
Agriculture 21 11 12% 12* 28 15 31 (8) 
Construction 9 15 9% 6 20 32 4 (20) 21* (2 
) 


Sales and 33 43 33% 38* 13 35 33 (53 
catering 


Other 37 31 45% 43* 40 18 30 (20) 28 (20)* 16 O 


Sources: NEPIA survey (March-April 2003) and INE (EPA Q1/2003) 

Note: Categories regarding sectors of activity have been adjusted to reflect published EPA 
data. NEPIA results regarding ‘other’ sectors of activity include interviewees (4.5 per cent of 
the total) who declined to respond. Regarding origin, available EPA categories refer to Latin 
America, Eastern Europe, the whole of the African continent and a residual ‘Asia plus 
Oceania’; since the latter is numerically irrelevant here, the NEPIA category ‘Asia’ has been 
maintained. EPA data with doubtful statistical representativeness are shown in italics and 
data relative to African respondents are in parentheses. In NEPIA, the reference month for 
employment-related questions was February 2003. Values (A) and (B) that do not differ sig- 
nificantly from one another are marked with an asterisk. All figures displayed are rounded. 
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Table 4.6 Employment qualification level, NEPIA sample (A) and corresponding 
nationalities in Andalusian subsample of EPA (B) (per cent) 


Qualification level (A) (B) 
1 — directors and managers 6 8 
2 — highly qualified professionals and technicians 4* 1* 
3 — complementary professionals and technicians 4 7 
4 — administrative staff 3* 4* 
5 — workers in catering, sales and security services 24 20 
6 — qualified agricultural workers 1 2 
7 — qualified workers in industry, mining and construction 7 18 
8 — machinery operatives 2 0 
9 — unskilled employment 49 41 


Sources: NEPIA survey (March-April 2003) and INE (EPA Q1/2003) 

Note: In NEPIA, the reference month for employment-related questions was February 2003. 
For lack of statistical significance in most subcategories, EPA data are not disaggregated by 
area of origin. Values (A) and (B) that do not differ significantly from one another are 
marked with an asterisk. All figures displayed are rounded. 


for statistical similarity. As to the discrepancies regarding certain other pro- 
portions, NEPIA data can be rated as at least as trustworthy as the EPA”s. 
For example, it is reasonable to assume that the share of entrepreneurs 
among Asian immigrants indeed exceeds that of any other group of origin. 

NEPIA’s data cover a range of groups of origin and sectors of activity 
for which no reliable EPA data are available. Actually, original measure- 
ment by NEPIA is far more detailed than shown here (Pérez Yruela & 
Rinken 2005). Apart from its added value, that information can generally 
be rated as credible; for example, as sizeable numbers of East European 
and African employees in Andalusian agriculture have been widely docu- 
mented. The relatively low proportion of NEPIA interviewees who were 
employed in sales and catering should be interpreted as a positive sign re- 
garding data accuracy, since for fieldwork staff, this is an easy-access em- 
ployment category; visibility bias in NEPIA would thus have caused the 
share of this category to be greater than that of the EPA. The much larger 
share of “other” employment sectors in all of NEPIA’s groups of origin, as 
compared to EPA results, again can be seen as evidence of well-balanced 
NEPIA interviewee recruitment procedures. Specifically, NEPIA fieldwork 
proved capable of capturing less visible and even illicit activities. 

Finally, albeit missing the test for statistical coincidence in most cases, 
NEPIA results on migrants’ employment qualification levels again reveal 
considerable similarity with EPA data. Both surveys put the share of un- 
skilled employment at between 40 and 50 per cent, the share of employees 
in catering and sales services at between one quarter and a fifth of the total, 
and the proportion of white-collar jobs (levels 1-4), at 20 and 16.5 per 
cent, respectively. The differences between both sources do not imply, in 
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principle, any particular bias. The most noteworthy discrepancy regards 
workers in the construction industry, which accounts for most employment 
in group 7. It is conceivable that construction workers were less accessible 
for on-site recruitment by NEPIA interviewers than other types of employ- 
ees. Still, one fifth of NEPIA interviewees from Eastern Europe and sub- 
Saharan Africa did report being employed in the construction industry (see 
Table 4.5), which implies that NEPIA fieldwork did manage in principle to 
approach this segment of the migrant labour force. Possibly, there is some 
mismatch in EPA between the proportion of skilled construction workers 
and unskilled jobs in this and other industries (group 9), respectively. 

To put these comparisons in perspective, it is worth noting that the EPA 
estimate of about 85,000 migrants from less developed countries working 
in Andalusia in the first quarter of 2003 is about 45,000 units short of an 
educated guess based on NEPIA magnitudes.” 


4.5 Summary and conclusions 


Faced with a demanding deadline and complex challenges regarding 
research design and implementation, the NEPIA survey succeeded in col- 
lecting a host of largely sensitive data regarding the social situation of im- 
migrants across Andalusia, which is itself a large and varied region. 
Avoiding the under-coverage that would have resulted from using an ad- 
ministrative register as the sole basis of interviewee recruitment, data were 
obtained from a socially and culturally diverse spectrum of people. 
Methodologically, the NEPIA’s most innovative aspect was the decision to 
sample interviewee search profiles — thus it disconnected general fieldwork 
logistics and data weighting, to be based on available administrative mag- 
nitudes, from the recruitment of interviewees matching those profiles, 
which was to be achieved by fieldwork staff. This decision was risky, mak- 
ing data collection potentially vulnerable to visibility bias. However, ex- 
cept for minor qualms that mostly affect the Asian sub-sample, external 
validation essentially indicates satisfactory data quality. 

A gamble it was, though. Reflecting on the NEPIA experience today, 
with a view to drawing lessons that other researchers may wish to take into 
consideration, it is worth pointing out that the intense and continuous effort 
on the part of the research team was just a prerequisite, rather than a guar- 
antee, for successfully completing the survey. The data-gathering operation 
may well have failed even despite the incessant attention paid to every de- 
tail of the research process. Hence, if available at all to researchers operat- 
ing in highly dynamic demographic settings, we would generally advise 
making use of more traditional sampling procedures. Apart from a less 
challenging resource portfolio, from the research team’s perspective, stand- 
ard procedures have the advantage of externalising responsibility for the 


ENHANCING REPRESENTATIVENESS IN HIGHLY DYNAMIC SETTINGS 105 


limitations of the data obtained: if the material validity of survey results 
are found to be hampered by a restrictive sample frame, the fault would 
surely lie with the sources at hand to construct the frame, rather than the 
peculiarity of researchers” decisions. The attempt to find creative solutions 
is risky by definition, since it relocates the burden of proof regarding data 
accuracy; any suspicion of systemic error may cast a long shadow. By con- 
trast, random error tends to be computed meticulously with regard even to 
sample frames in which comprehensiveness cannot be assessed with any 
degree of precision. In the case of immigrant populations, there may ac- 
tually be evidence of under-coverage far exceeding the margin of error that 
is generally deemed acceptable in survey methodology. 

NEPIA’s general lesson, then, is to avoid added risk when possible but 
to take risks when needed, that is, when operating in an empirical setting 
that defies the possibility or plausibility of standard probability sampling. 
If pressed to assess which of NEPIA’s key decisions may reasonably be 
emulated in similar circumstances, the idea of sampling interviewee search 
profiles still seems attractive to us, as does the cross-stratification of the 
sample and the obligatory diversification of interviewee recruitment points. 
However, with hindsight, we would think even harder about the possibility 
of relying on professional interviewers, rather than culturally and linguisti- 
cally ‘streetwise’ amateurs, for the crucial step of actually recruiting and 
interviewing survey participants. This is mainly due to misgivings regard- 
ing an insufficient degree of control on the part of researchers over the 
NEPIA’s culturally and linguistically very diverse data-collection operation. 
As observed by DaVanzo, Hawes-Dawson, Burciaga Valez & Vernez 
(1994: 59), ‘the optimal approach to recruiting, training and supervising a 
large multicultural interviewing team is to use a multicultural (and experi- 
enced) supervisory team’, a condition that would have been extremely hard 
to meet even with a less challenging deadline. This is due to the enormous 
difficulty of finding suitable supervisory staff. The fact that the NEPIA 
successfully recruited interviewees from some eighty countries, as opposed 
to the two nationalities targeted by DaVanzo and associates, highlights the 
magnitude of that difficulty. Technology available today would perhaps al- 
low a translation service to be established, accessible by mobile phone, in 
order for multi-linguistic competence to be a less vital criterion for select- 
ing fieldwork staff. That said, the capacity of building interpersonal rapport 
is truly essential. On this count, the jury is out as to whether interviewers 
sharing the status of immigrants — albeit not necessarily the interviewee’s 
particular ethnic group — indeed do better than native fieldwork professio- 
nals, as we assumed when recruiting fieldwork staff for the NEPIA survey. 

Resource requirements imply that NEPIA-style surveys are unlikely to 
be repeated periodically, which basically limits them to one-off special da- 
ta-gathering events. However, this is probably true for any sizeable special- 
population survey, including those with relatively classic probability 
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samples. This may point to oversampling in general population surveys as 
the “best-value” data-collection method, provided that limitations regarding 
coverage are deemed acceptable. 

In sum, when assessing the risk-benefit profile of NEPIA-style sampling 
and fieldwork procedures, it seems appropriate to do so not in an abstract, 
wish-list sort of way, but rather with a view to its alternatives — or lack 
thereof — in a given empirical setting. To push this point a little further, 
would we opt for NEPIA-style procedures again if asked to repeat a similar 
survey? The timeliness and inclusiveness of the ‘Padrón’ population regis- 
ter has improved markedly throughout the past five years, turning its statis- 
tics into a rather credible candidate for serving as sample frame for a sur- 
vey on Spain’s immigrant population. There are still issues regarding this 
sources degree of coverage. We have mentioned the exclusion of people 
in precarious housing situations, which are relatively common among im- 
migrants in some parts of Andalusia. For similar reasons, undocumented 
migrants are susceptible to being covered in less-than-accurate proportions. 
Nonetheless, the equation has clearly changed since 2003, when the 
NEPIA survey was conducted. At this point in time, we believe that the 
risk-benefit profile of turning the “Padrón” into the outright sample frame 
for a migrant survey — as opposed to sampling search profiles, as in 
NEPIA — depends chiefly on the specific research objectives. This is to 
say, ‘Padron’-based probability sampling would, in our view, be adequate 
for research essentially on migrants’ social networks (as is the case with 
the ENI), for example, but perhaps not for a study aiming at gathering reli- 
able data on the migrant population’s relationship with the host society’s 
institutions and administrative procedures. For all the benefits of ‘pure’ 
probability surveys, sampling on the dependent variable (i.e. “Padrón” 
registration) still appears to us to constitute a major drawback. 

Given the exceptional status of Spain’s “Padrón” from a comparative 
point of view, migration researchers in many parts of the world are surely 
unable to identify any data that could be used plausibly as a full sample 
frame, as was the case with NEPIA in 2002. Therefore, we believe the 
NEPIA approach may prove useful wherever administrative data allow one 
to reasonably gauge the migrant population’s two most basic traits: compo- 
sition (in terms of country or region of origin) and geographical distribu- 
tion (in terms of province or municipality). As long as immigrants who are 
excluded from the data are not distributed geographically in ways system- 
atically different from the settling patterns of immigrants that are included 
(and provided the search profiles based on that proxy sample can be imple- 
mented properly), the data’s degree of coverage relative to the target popu- 
lation ceases to be a pressing concern. In short, sampling interviewee re- 
search profiles entails advantages that may outweigh the — considerable — 
risks and added difficulties. 
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Notes 


1 The NEPIA survey was co-funded by the regional government of Andalusia (specifically, 
the Directorate General for the Coordination of Migration Policies, pertaining to the 
Ministry of Governance at that time) and the European Social Fund. The study’s coordi- 
nator — and author of this chapter — was then a beneficiary of the “Ramón y Cajal” pro- 
gramme sponsored by the Spanish Ministry of Science and Technology. Working closely 
with the NEPIA's research team, technical staff at the Institute for Advanced Social 
Studies (IESA-CSIC) contributed vitally to the successful completion of the study. For de- 
tails see Rinken (2003) or Pérez Yruela and Rinken (2005). 

2 The NEPIA acronym refers to the Spanish wording for “Needs of Andalusia’s Immigrant 
Population’ (NEcesidades de la Poblacion Inmigrante en Andalucia). 

3 The author, who coordinated the NEPIA study, is grateful for this opportunity to reflect 
on the research process and outcomes. As about five years have passed since publishing 
most NEPIA-related work, the balance of involvement and detachment seems right for a 
fresh appraisal. Shortly after completion of the study, the Spanish journal ‘Metodologia 
de Encuestas’ (Survey methodology) dedicated a thematic dossier specifically to discus- 
sing NEPIA’s advances in enhancing representativeness (see Maya Jariego 2003; Rinken 
2003; Gualda, Palacios, Teves & Rinken 2003). 

4 With hindsight, the odds that foreigners living in Spain would proceed with their “Padrón” 
registration were substantially increased by a law passed in the year 2000, which guaran- 
teed access to public health care and education to any person registered, regardless of oth- 
er administrative situations (Aja & Arango 2006). 

5 ‘Pateras’ are small and generally precarious boats used for illegal crossings of the Strait, 
as opposed to the more sizeable ‘cayucos’ employed for the route from Western Africa to 
the Canary Islands. 

6 Iam grateful to Gema Galera and Pilar Cortés for their help in compiling the data used in 
this section. 

7 Data in tables 4.2 through 4.6 have been tested for statistical significance at a 95 per cent 
confidence level. 

8 Encuesta de Poblacion Activa. 

9 Adding 25 per cent for immigrants not registered in the ‘Padrón’ to the quantification of 
NEPIA’s proxy-universe of 148,500 people, the reported employment rate of 71 per cent 
generates an estimate of about 132,000 migrant workers as of February 2003. 
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PART II 


FIELDWORK AND RESPONSE RATES 


5 The influence of interviewers’ ethnic 
background in a survey among Surinamese 
in the Netherlands 


Anja van Heelsum 


5.1 Introduction 


This chapter examines whether there are different results due to the ethnic 
backeround of the interviewer in a survey on ethnicity among second-gen- 
eration Surinamese carried out in Amsterdam. 

The literature on interviewer effects is part of a larger body of work on 
response effects. Section 5.2 first reviews response effects in general and 
then looks at research on the influence of the ethnic background of the in- 
terviewer in survey research (in the Anglo-Saxon literature simplified to 
race of interviewer effect). Attention is given to the respondents, the sub- 
ject matter of the questions, methods and explanations for this phenomen- 
on. Section 5.3 describes the research design of the survey of 300 second- 
generation Surinamese. Section 5.4 looks at the outcomes of this study us- 
ing first a method based on individual scores and then multi-level analysis. 
Finally, section 5.5, the conclusion, discusses the relevance of the out- 
comes in terms of the theories that are found in the literature. 


5.2 Existing research on response effects and race of 
interviewer effects 


Response effects 


Van der Zouwen (1989: 9) defines “response effects” as those particular ef- 
fects on the response that are caused by variables other than the ones 
which the researcher wants to measure with the stated question. Dijkstra & 
Van der Zouwen (1982) refer to four types of variables that can cause re- 
sponse effects. First, they distinguish variables that are related to the char- 
acteristics of the interviewer, which can include role dependent and role in- 
dependent characteristics. Role dependent characteristics are, for example, 
motivation and skilfulness in interviewing in the intended manner. Role in- 
dependent characteristics include, among others, ethnic background, sex, 
age and other personal characteristics of the interviewer, but in addition, 
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they also include the interviewer's own expectations and opinions regard- 
ing the subject matter of the survey and respondents” answers. 

The second type of variables relates to characteristics of the respondents. 
These include role dependent characteristics, such as the motivation to an- 
swer honestly, and role independent ones, for example, ethnic background, 
sex and age, but also psychological variables such as the tendency to an- 
swer in a socially desirable way. 

Third are variables related to the interview situation, like the extent of 
anonymity, the place and time of the interview, and also whether third per- 
sons are present, in front of whom the respondent may not feel free to say 
what he or she wants or may be distracted. 

Finally, there are variables related to the characteristics of the questions 
or the questionnaires. Molenaar (1986) distinguishes between the content 
and the form of the questions and questionnaire. The content of the ques- 
tions can be of greater or lesser importance to the respondents. The re- 
spondents can experience the content as threatening to their privacy, as 
may occur with questions about personal relations, money and criminality. 
An example of a form characteristic of questions that influence answers is, 
for instance, the extent to which respondents can formulate their answers 
in their own words. Multiple choice items limit the alternatives to those 
formulated by researchers in advance, while an open question can lead to 
more diverse answers. A form characteristic of a questionnaire that can in- 
fluence the answers is the order of the questions. Asking multiple choice 
questions on a subject before an open question on that same subject may 
guide respondents in a certain direction, and they may only think of what 
was already suggested in the multiple choice section. Such manipulation is 
not possible when the open question is asked first. The influence of the 
way questions are formulated was studied by Molenaar (1986). Suggestive 
question formulations were studied by Smit (1995), and specific problems 
with retrospective questions were studied by Van der Vaart (1996). 


The influence of the ethnic background of the interviewer 


The influence of the ethnic background of the interviewer refers to all sys- 
tematic differences in the answers of respondents which arise when inter- 
viewers from different ethnic backgrounds pose exactly the same ques- 
tions. When systematic differences are found, it is not automatically clear 
which of the ethnic groups of interviewers caused a misrepresentation and 
which group obtained correct results. The US literature often assumes that 
results obtained by interviewers from the same ethnic group as the re- 
spondents will be nearer to the truth, while chance of bias increases when 
the interviewer is from another ethnic group (for an example, see Schaeffer 
1980). Rhodes (1994) criticises this assumption and I am in agreement 
with him that this assumption is difficult to prove, except for factual 
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questions. In this study I investigate in which cases Surinamese and Dutch 
interviewers obtain different results and how these differences can be 
explained. 

Before looking at the results of the survey, I will provide a short over- 
view of the literature on the impact of the ethnic background of the inter- 
viewer. First I will look at the question of whether systematic differences 
actually occur when interviews are carried out by interviewers from differ- 
ent ethnic groups. The answer to this question is both yes and no: some re- 
searchers have found differences between two interviewer groups and 
others not. Some examples can be mentioned: Baratz (1967) found differ- 
ences in outcomes on test anxiety and stress depending on whether the in- 
terviewer was African American or white; Berk and Bernstein (1988) 
found no differences on medical questions; Esbensen and Menard (1991) 
concluded that there were no differences in answers on delinquency and 
male-female stereotypes; and Campbell (1981) found no differences in a 
study among young adults on neutral and political subjects but did find dif- 
ferences on race-related topics. 

If systematic differences do exist, the next question is: do they occur 
more often with certain respondents, with certain subject matters and with 
certain research methods? The last question is why do systematic differen- 
ces occur? 


Among which respondents do systematic differences occur? 

Much research on the “race of interviewer effect” has looked at African 
American (black) and white respondents in the United States (Williams 
1964; Bryant, Gardner & Goldman 1966; McClelland 1974; Campbell 
1981; Cotter, Cohen & Coulter 1982; Edwards 1990; Finkel, Guterbock & 
Borg 1991).' In general, systematic differences have been found in the re- 
sponses given by black and white respondents when the same questions 
are asked depending on whether the interviewers are white or black, 
though it always depends on the subject matter of the questions. A few 
studies report a stronger “race of interviewer effect” among white respond- 
ents than among black respondents (Bryant et al. 1966, Cotter et al. 1982, 
Finkel et al. 1991). Bryant et al. (1966) concluded, for instance, in their 
study on inter-ethnic relations, that white respondents give more stereo- 
typed answers about other ethnic groups when interviewed by an inter- 
viewer of their own ethnic group than do black respondents. 

Less systematic research has been done in the United States regarding 
other racial or ethnic groups. Baca Zinn (1979), Weeks and Moore (1981) 
and Reese, Danielson, Shoemaker, Chand and Hsu (1986) studied 
Mexicans. Leslie, Raglin and Schwede (2002) found differences in the re- 
sults from Hispanic and white interviewers in the US Census. Baca Zinn 
(1979) — as an exception — did not use a survey method and argued that an 
interviewer of Mexican origin has more insight into the motives of 
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Mexican respondents. Because these respondents felt understood, they pro- 
vided more confidential information. Weeks and Moore (1981) found no 
influence of the background of the interviewer, while Reese et al. (1986) 
found differences between English- and Spanish-speaking respondents 
when interviewers were either from their own or from another ethnic 
group. 

Meloen and Veenman (1990) studied response effects in surveys among 
immigrants in the Netherlands. The black/white distinction was not used 
here, but rather, interviewers who were from the respondents’ own group 
were distinguished from those who were not. Generally, they found that 
the background of the interviewers had some influence, but it is not always 
clear to what extent this can be generalised. Some of the researchers whom 
they quote had not used the survey method, and therefore did not consis- 
tently use the same questions and the same conditions. Risvanoglu-Bilgin, 
Brouwer and Priester (1986) concluded in their study among Turks in the 
Netherlands — in which two female interviewers, one Turkish and the other 
Dutch, worked with a questionnaire with open and closed questions — that 
the Turkish interviewer gathered more information with the open questions 
than the Dutch interviewer. These findings correspond with those of Baca 
Zinn (1979). More publications reporting systematic survey research on the 
effect of the ethnic background of the interviewer in the Netherlands has 
come out since the fieldwork for this study was completed, such as the 
work of Van’t Land (2000, 2002) who studied attitudes towards doctors 
and Dotinga, Van den Eijnden, Bosveld and Garretsen (2004) who studied 
alcohol use among Turks and Moroccans. Attitudes towards doctors were 
more positive and reported alcohol consumption was higher when a Dutch 
interviewer spoke to respondents than when a Turkish or Moroccan inter- 
viewer talked to them. 


In what subjects do systematic differences occur? 

As mentioned, the subject matter of the interview is relevant when study- 
ing ‘race of interviewer’ effects. Davis (1997) states that with most sub- 
jects, including factual and attitudinal matters, such an effect may occur. In 
his study, on 60 per cent of the attitude questions related to the US elec- 
tions and the candidacy of Jesse Jackson, black respondents adjusted their 
response to what they assumed were the expectations of the interviewer. 
Subjects like racial consciousness and support for Jesse Jackson were 
clearly more biased, but so too were even factual questions on the electoral 
system. 

Following other authors, I distinguish ethnic-related subjects — in the 
United States called ‘racial’ — from subjects that are unrelated to anything 
ethnic or racial (Schuman & Converse 1971; Sudman & Bradburn 1974; 
Campbell 1981; Weeks & Moore 1981; Cotter et al. 1982). Ethnic-related 
subjects are issues that the respondent can associate with membership of 
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any ethnic group. Examples are ‘what do you think about Turks’ and ‘do 
you read Turkish newspapers”. Campbell (1981) found that the background 
of the interviewer had greater influence on ethnic-related subjects than on 
non-ethnic ones. Other authors report only on ethnic-related subjects. 
Hatchet and Schuman (1975), for instance, used a proximity scale for 
white American respondents. It included questions such as “would you 
marry a black person”, ‘would you let your children go to a school with 
black children” and “would you live on a block where blacks also live”. 
The influence of the ethnic background of the interviewer has been clearly 
demonstrated in these studies. White Americans answer more positively 
with a black interviewer when asked about contact with black Americans 
than they do with a white interviewer. 

Kelley, Hovland, Schwarz and Abelson (1955) found similar results 
when asking white Americans about their attitudes towards blacks: atti- 
tudes were more positive when a black interviewer asked the questions. 
Ethnic-related subjects do not always provoke interviewer effects, though. 
Bryant et al. (1966) were surprised that the interviewer effects were ac- 
tually limited in the aforementioned study on stereotypes. Cotter, Cohen 
and Coulter (1982) also concluded that interviewer effects can be found in 
some but not in all ethnic-related subjects. They found an interviewer ef- 
fect among white respondents with certain ethnic questions. 

Some subjects are indirectly related to ethnic issues or membership of 
an ethnic group. According to Schaeffer (1980), the subject “welfare bene- 
fits” 1s associated with certain ethnic groups in the United States. Because 
of this association, an (indirect) “race of interviewer” effect can be shown. 
Williams (1964) provided another example: the question ‘do you read a 
newspaper’ seemed to provoke ethnic associations, since in 1964 black 
Americans reading newspapers was seen as a sign of their activism. 
Williams stated that the extent to which a subject is threatening or sensitive 
in relation to ethnic relations determines the extent of the bias. 

Van’t Land (2000, 2002) showed that when the subject matter was fac- 
tual there was no effect of the Moroccan background of the interviewer 
who interviewed second-generation Moroccan youngsters in the 
Netherlands, but differences were found when the subjects were related to 
ethnicity and with subjects that left greater room for subjective answers. 
Biliet and Loosveldt (1988) had already suggested that interview formats 
which leave more room for ‘autonomous interviewer activity’ also leave 
more room for positive and negative feedback in the interview process, 
and thereby more room for the influence of the interviewer. 

Beside the distinction between ethnic and non-ethnic subjects, a number 
of authors distinguish between personal and non-personal issues. An exam- 
ple of a personal issue is partner relationships (Weller & Luchterhand 
1968). A related issue is questions that may somehow be threatening 
(Williams 1964). Interviewers have greater influence on answers when the 
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questions are considered personal or threatening. “Race of interviewer” ef- 
fects were not reported when studying the size of the personal network of 
respondents with name generation methods, though the educational level 
of the interviewer and the cooperativeness of the interviewee increased the 
number of names generated (Van Tilburg 1998; Marsden 2003). 

Interviewer effects were also found with medical-related subjects (Berk 
& Bernstein 1988) and voting behaviour (Anderson, Silver & Abramson 
1988a; Anderson, Silver & Abramson 1988b; Finkel, Guterbock & Borg 
1991). The advantage of these subjects is that there is a ‘true answer’ 
which can serve to validate the answers given to black or white inter- 
viewers. One can see with which interviewer more accurate answers are 
given and with which interviewer, more biased answers are obtained. 
External validation is not possible for attitude questions. 


What research methods cause more systematic differences? 

Much of the research referred to above used the face-to-face survey inter- 
view method. This method is suitable for research on the ethnic back- 
ground of the interviewer, since the number of interviewers is large enough 
for a systematic comparison to be made, and the questions are the same for 
every interviewer. But effects of the background of the interviewer have al- 
so been shown in telephone surveys. 

Cotter, Cohen and Coulter (1982) demonstrated that respondents drew 
conclusions about the race of the interviewer based on his or her accent 
and speech pattern and adjusted their answers accordingly. This study took 
place in the United States and included black and white interviewers. 
Campbell (1981) showed that this effect might even occur when written 
questionnaires are used distributed by researchers of different backgrounds. 
Edwards (1990) found differences in unstructured interviews; and Baca 
Zinn (1979) found an interviewer effect when using participant observation 
among Mexicans. 

An important aspect of the research method is the form in which a ques- 
tion is posed. A difference may, for instance, occur between interviewer ef- 
fects in open questions and multiple choice questions. To distinguish the 
question form analytically from the subject matter of the question, a design 
has to be used in which the subject remains constant per question form. In 
the literature no comparison can be found on interviewer effect between 
open and closed questions on the same subject. Anderson, Silver and 
Abramson (1988a, 1988b) posed a number of questions on warmth and 
closeness towards black Americans. Surprisingly, they found differences 
between black and white interviewers when asking questions on closeness, 
but not on warmth. The authors concluded that the subject could not be the 
reason for the different results. Only the formulation of the questions and 
the answer alternatives could provide an explanation. Based on these 
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results no clear conclusions can be drawn about research method or ques- 
tion form. I will therefore not use it in my own study. 

A new development is the use of virtual interviews via the Internet. 
Krysan and Couper (2006) report “race of interviewer” effects using com- 
puter-assisted self-interviewing (CASI), in which videos were shown of an 
interviewer reading the questions. The answers on a prejudice scale were 
influenced, but unexpectedly, racist answers became worse with a black 
interviewer. 


Why do systematic differences occur? 

A number of explanations can be found for the different results obtained 
from interviewers from different ethnic groups. The first and most common 
one is the social desirability explanation (Finkel et al. 1991). Campbell 
(1981) used the term deference and Anderson, Silver and Abramson 
(1988b) talked about ‘politeness towards a stranger’. Because of respect, 
consideration or politeness towards the interviewer, respondents adjust or 
moderate their answers. Campbell concluded that both black and white re- 
spondents tend to talk ‘in the direction’ of the interviewer. Black respond- 
ents assume, for instance, that white respondents do not like to hear that 
discrimination is common. Gong and Aadland (2006) and Loureiro and 
Lotade (2004) tested the social desirability hypothesis on willingness to 
pay for environmental goods and Fair Trade coffee from Africa. The will- 
ingness to pay more for coffee from Africa was nearly double with African 
interviewers than with white interviewers, suggesting that respondents 
wanted to please the interviewer (Loureiro & Lotade 2005). 

Edwards’ (1990) explanation was that mistrust can exist in inter-racial 
situations, especially with black respondents and a white interviewer. 
Segregated relations, as in the United States, but also a colonial past and 
slavery can be reasons for mistrust between blacks and whites. A similar 
reasoning is found in the work of Dutch authors, such as Mullard, Nimako 
and Willems (1990) and Essed (1986). They argued that existing power re- 
lations disturb communication between black respondents and white inter- 
viewers to the extent that it is not possible to get valid answers from black 
respondents. 

Edwards (1990) added the fear of black respondents that data will be 
abused by white interviewers and the lack of understanding of racism by 
white interviewers. Black respondents may assume that a white interviewer 
will not understand their accounts of discrimination, having never suffered 
it. This third explanation is related to the mistrust explanation, but more fo- 
cused on questions associated with negative inter-ethnic relations. This ex- 
planation assumes that the ‘race of interviewer’ effect occurs only when 
questions concern negative inter-ethnic relations like racism. 

Weller and Luchterhand (1968) thought that the general political situa- 
tion might boost the interviewer effect. Their research in a US town took 
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place during a period of racial unrest, and the interviewer effect was stron- 
ger than it would have been in a relatively peaceful period. Race relations 
were not, at that point in time, a neutral subject, and in the interview the 
race of the interviewer interfered more than when racial issues were not in 
the news. In such periods relationships between ethnic groups can be pre- 
carious. Neither interviewer nor interviewee feel at ease, as they may as- 
sume that the person they are speaking with holds the more extreme atti- 
tudes of people they see in the news. Such a situation occurred with 
Muslims after the incidents of 9/11 (Harchaoui & Huinder 2003). 

The next explanation focuses on the extent to which respondents esti- 
mate that their information will be kept confidential: the anonymity explan- 
ation. Supporters of this explanation assume that the data gathered by in- 
terviewers from their own ethnic groups are biased, since respondents fear 
that their story may circulate in their community. Fear of gossip is prob- 
ably stronger in small ethnic communities, but Weller and Luchterhand 
(1968) also noted this phenomenon among black Americans (a large 
community). 

The last explanation, based on the theory of stereotype threat, assumes 
that respondents from groups that are often the victim of negative stereo- 
typing, for instance, black respondents in the United States, become nerv- 
ous with and give more incorrect answers to a white interviewer, even to 
factual questions (Davis & Silver 2003). This theory was developed based 
on intelligence data and athletic performance, but Davis and Silver (2003) 
also demonstrated this effect in questions on political knowledge. 

The explanations that I have listed are often formulated in the context of 
the United States. Of course there are important differences between the 
Netherlands and the United States regarding relations between ethnic 
groups. Relations between African Americans and whites in the United 
States have been an institutionalised problem for ages, with slavery and the 
civil rights movement in their history. Strong segregation in cities has also 
long been characteristic of the United States. In the Netherlands, less atten- 
tion 1s given to racial differences and more attention to religious differen- 
ces, which may lead to stigmatisation of groups such as Muslims. 
Immigration of many ethnic groups to the Netherlands began only in the 
1960s. It is possible that relations between ethnic groups in the 
Netherlands are less static and more flexible than in the United States, so 
concerns such as mistrust and lack of understanding are less prominent 
among immigrants in the Netherlands. 

In fact, the explanations presented above do not exclude one another; 
rather, they may supplement one another. It is surprising to find that most 
of the explanations consider the data gathered by black interviewers as 
nearer to the truth and that from white interviewers as more biased. 
Rhodes (1994) remarked that there is seldom a clear-cut ‘right’ answer. 
Though this explanation seems probable when we peruse the subjects and 
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issues on which the disparity occurs, validating information on attitudinal 
questions is available only in the case of political knowledge and voting 
behaviour. 

The design of the research presented in this chapter takes into considera- 
tion the issues raised by these studies. Because ethnic-related and personal 
subjects are easily influenced by interviewer effects, these two themes are 
the focus in accordance with the following mechanisms: 1) ethnic-related 
subjects in which mistrust and social desirability might cause interviewer 
effects; 2) personal topics, where social desirability is the most likely ex- 
planation for interviewer effects. The study, moreover, investigates the as- 
sumption that interviewer effects occur more with ethnic-related subjects 
and personal subjects than with matters that have nothing to do with these 
topics. 


53 Research design 


This study was designed in the following manner. Firstly, for a thorough 
operationalisation of the theoretical concepts, two pilot studies were organ- 
ised. Open interviews took place with twelve second-generation 
Surinamese. These were meant as a first test of the questionnaire focusing 
on content. In a second pilot study a postal questionnaire was sent to 600 
people, of whom 225 persons (38 per cent) replied. The second pilot 
served to further improve the questionnaire, to refine the question sets and 
to validate the scales. 

The resulting final questionnaire was used in the main survey, which 
consisted of face-to-face interviews. Addresses were randomly selected 
from a database received from the population registry of the Municipality 
of Amsterdam of 4,390 persons aged between 15 and 35 years and with at 
least one parent who was born in Surinam.” From this list, 525 respondents 
were approached at home, of whom 300 (58 per cent) actually took part in 
interviews. The interviews were randomly assigned to Surinamese and 
Dutch interviewers. Though most Surinamese interviewers were recognis- 
able because of their darker skin colour, I broadened the simplistic contrast 
between black and white interviewers found in the American literature to 
perceptions of an ‘in’ group and an ‘out’ group. Surinamese interviewers 
can be considered members of one’s ‘own’ group, and Dutch interviewers 
members of the ‘other’ group. This seems a more nuanced way to opera- 
tionalise how the ethnic background of the interviewer might be conceived. 
The assumption is that it is not so much the colour of the interviewer’s 
skin that determines the contrast between Surinamese and Dutch inter- 
viewers, but the perception of whether the interviewer belongs to one’s 
‘own’ group (it can be matter of accent, hair, use of certain words, but also 
what the interviewer says about him or herself). 
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Data from interviewers who gathered fewer than ten interviews have 
been left out, so in the analyses the data collected by sixteen interviewers 
are used, eight of whom were Dutch (151 respondents) and eight 
Surinamese (149 respondents). In both cases (Dutch and Surinamese inter- 
viewers) there were seven women and one man. The educational level of 
the interviewers was similar since all were university students. 


Selected item sets 


Seven measurement instruments were selected for testing interviewer ef- 
fects. Since I expect greater differences with the ethnic-related subjects, 
followed by the personal subjects, than neutral subjects, I selected the 
following: 

a. at least one measuring instrument that can be considered to measure re- 
sponses to questions regarding an ethnic-related subject; 

b. at least one instrument that measures responses to questions regarding a 
personal issue; 

c. at least one measuring instrument that can be considered to measure re- 
sponses to questions regarding both an ethnic-related subject and a per- 
sonal subject; 

d. at least one measuring instrument that does not measure responses to 
questions regarding either ethnic-related or personal subject matters. 


Following the reasoning in section 5.2, I expect that the interviewer effect 

will decrease in the order c, a, b, d, as I expect that the combination of two 

issues sensitive to interviewer effect — ethnic-related plus personal subjects 

— will result in the largest difference between Surinamese and Dutch inter- 

viewers. 1 expect less difference between Surinamese and Dutch inter- 

viewers with just ethnic-related subjects, and even less with personal sub- 
jects. The smallest differences or possibly no difference at all is expected 
when a subject is neither ethnic-related nor personal. 

Ideally, scales and measuring instruments should not be sensitive at all 
to interviewer effects, but as the literature review has shown, it is too opti- 
mistic to assume that this would be the case. I developed a number of 
scales in the category ethnic-related subjects (Van Heelsum 1997: 82-106). 
Overall, the following validated measuring instruments will be used: 

— In the category ethnic-related subjects, four instruments have been 
selected. The first is a scale on ethnic self-definition; with five multi- 
ple choice items. The second is a scale on orientation towards the 
Surinamese language and culture. The third is a scale on orientation 
towards contact with Surinam via post and telephone. A scale on 
perceived position allocation is the fourth instrument. Ethnic self-def- 
inition, language and culture and position allocations are attitudinal 
subjects, while contacts through post and telephone are more of a 
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behavioural measure based on the actual number of letters and phone 
calls. 

— In the category personal subjects, I have chosen the loneliness scale 
(De Jong-Gierveld & Kamphuis 1985), which consists of eleven multi- 
ple choice items. 

— In the category ethnic-related subjects and dealing with personal sub- 
jects I have selected a scale on orientation towards contacts with the 
Surinamese group. 

— In the questionnaire there is no “neutral” subject that is measured with a 
scale. Therefore I will use the measure of educational level which con- 
sists of one question with sub-questions. 


Multi-level analysis 


It is possible to analyse the interviews on an individual basis, but also on a 
group basis per Interviewer. In the past I have used an analysis of variance 
to study this kind of data (Van Heelsum 1993). But calculating a mean per 
interviewer for his or her respondents, or attaching an interviewer score to 
all respondents causes loss of information, and this results in loss of data 
and even completely erroneous conclusions. Goldstein (1996) demon- 
strated this when he analysed school classes, distinguishing school varia- 
bles, class variables and student variables. 

Multi-level analysis is a technique that takes into account that data are 
gathered at different levels, in this case, at the level of respondents and at 
the level of the interviewer. Following Hox, De Leeuw and Kreft (1991), 
Hox (1994) and Van Tilburg (1998), who explained the advantages of us- 
ing multi-level methods, I will first present data at the individual level 
(mean scores) and then apply multi-level analysis, for which the computer 
program ML-3 is used (Prosser, Rashbash & Goldstein 1991). 


5.4 Results 
Results at the individual level (mean scores) 


An initial unexpected finding of the survey was that the second generation 
of Surinamese consists of a larger than anticipated number of individuals 
with only one Surinamese parent. The number of Surinamese parents is a 
variable that needs to be included in all further analysis, as it influences 
the scale scores considerably. Therefore, at least two means are needed to 
show interviewer effects, one for respondents with one Surinamese parent 
and one for respondents with two Surinamese parents. I present these mean 
scores to provide information comparing different subject matters and to 
determine which subjects are most sensitive to interviewer effects. Means 
and standard deviations of the selected scales are presented in Table 5.1, 
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grouped by both the background of the interviewee and by the background 
of the Interviewer. 

This is data at the individual (respondent) level. The ethnic-related sub- 
jects are presented in the table first (1-4), followed by subjects that are eth- 
nic-related as well as personal (5), by a subject which is only personal (6) 
and then subjects that are neither ethnic-related nor personal. The second 
column of Table 5.1 divides respondents into those with two Surinamese 
parents (S/S) and those with one Surinamese and one Dutch parent (S/D). 
In the third column, this group is again divided into those respondents that 


Table 5.1 Real and standardised mean scores and standard deviation of respondents 
with one or two Surinamese parents and a Dutch or Surinamese 


interviewer 
Measure Respondent’s Interviewer Mean S.D. Stand. Stand Difference 
parents mean s.d. Surinamese/ 
Dutch 
interviewer 
1. Ethnic self- S/S Sur (n=80) 3.2 1.4 6.4 2.8 0.5 
definition (0-5) Du (n=69) 2.9 1.4 5.9 2.9 
S/D Sur (n=46) 1.5 1.2 3.0 2.3 0.9 
Du (n=68) 1.0 1.2 2.1 2.3 
2. Orientation S/S Sur (n=80) 3.8 1.1 7.7 2.3 1.0 
Surinamese language Du (n=69) 3.4 1.5 6.7 3.0 
and culture (0-5) S/D Sur (n=46) 1.9 1.4 3.8 2.8 0.0 
Du (n=68) 1.9 1.4 3.8 2.8 
3. Contact in S/S Sur (n=80) 1.3 1.3 3.2 3.1 0.1 
Surinam (0-4) Du (n=69 1.2 1.3 3.1 3.1 
S/D Sur (n=46) 0.9 1.1 2.2 2.9 -0.1 
Du (n=68) 0.9 1.1 2.3 2.9 
4. Position allocation S/S Sur (n=80) 2.7 1.4 5.3 2.8 0.7 
(0-5) Du (n=69) 2.3 1.6 4.6 3.1 
S/D Sur (n=46) 1.7 1.2 3.4 2.5 -0.1 
Du (n=68) 1.7 1.4 3.5 2.7 
5. Orientation S/S Sur (n=80) 4.6 2.1 5.1 2.4 1.0 
towards contact Du (n=69) 3.7 1.9 4.1 2.1 
with the Surinamese S/D Sur (n=46) 3.2 1.9 3.4 2.1 0.6 
group (0-9) Du (n=68) 2.5 1.4 2.8 1.6 
6. Loneliness (0-11) S/S Sur (n=80) 2.2 2.5 2.0 2.3 0.8 
Du (n=69) 2.0 2.2 1.8 2.0 
S/D Sur (n=46) 1.8 2.5 1.7 1.7 0.7 
Du (n=68) 1.1 2.8 1.0 1.0 
7. Education (0-3) S/S Sur (n=80) 2.0 0.7 6.8 2.2 -0.7 
Du (n=69) 2.1 0.7 7.5 2.2 
S/D Sur (n=46) 2.4 0.8 8.0 2.5 0.7 


Du (n=68) 22 07 73 24 


Source: Van Heelsum (1997) 
a Last column shows the difference between standardised means of Dutch and Surinamese 
interviewers. 
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were interviewed by a Surinamese interviewer and those interviewed by a 
Dutch interviewer. The fourth and fifth columns shows the means and 
standard deviations. In the sixth and seventh columns the standardised 
means and standard deviations are given by transforming all scale scores 
into a 0-10 continuum. This standardisation enables comparisons to be 
made between scales on the Interviewer differences. In the last column, the 
standardised mean of Surinamese and Dutch interviewers per respondent 
group is presented. 

Table 5.1 shows that the mean outcomes of respondents with two 
Surinamese parents are higher on all scales than the outcomes of respond- 
ents with one Surinamese and one Dutch parent. The only exception is ed- 
ucational level, as the opposite is true there. Respondents with a 
Surinamese interviewer show a higher mean score than respondents with 
Dutch interviewers (positive difference) in most cases. Only on three scales 
are the scores of respondents with a Dutch interviewer higher (negative dif- 
ference). No univocal pattern is visible here. One can notice positive differ- 
ences between Dutch and Surinamese interviewers with both respondent 
backgrounds (full and half Surinamese) on (1) self-definition, (5) orienta- 
tion towards contacts with Surinamese and (6) loneliness. With the other 
scales we see nearly no difference between Surinamese and Dutch inter- 
viewers, for instance, on (2) orientation towards the Surinamese language 
and culture, (3) contacts in Surinam and (4) position allocation. 
Surprisingly in the case of (7) educational achievement, the background of 
the interviewer has the opposite effect in the two groups of respondents. 

In section 5.3, I formulated assumptions about the extent to which inter- 
viewer effects are expected based on various considerations. I will judge 
the correctness of these assumptions after the multi-level analysis. For now 
I will only remark that three scales show positive differences for both 
background groups, namely orientation towards contact in the Surinamese 
groups (ethnic and personal relations), the self-definition scale (ethnic) and 
loneliness (personal relations). With the other scales no systematic differen- 
ces occur. 


Results based on multi-level analysis 


The multi-level analysis takes into account that the data on respondents 
was gathered at a different level from the data on the interviewers. This 
procedure was also followed by Smit, Van den Eeden, Deeg and Beekman 
(1995) and Van Tilburg (1998). We controlled for the influence of back- 
ground, main daily activity, educational level and the age of the respond- 
ents on the scale scores. The analysis was done in three steps. The basis is 
a model with two levels, in which the scale scores of respondents and in- 
terviewer number are processed as independent variables, so that the (gen- 
eral) interviewer variance can be computed (Dijkstra 1983a, 1983b). 
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In the first step, gender, age, ethnic background, education and main ac- 
tivity of respondents are added as independent variables in the measure- 
ment model. In the second step, the ethnic background of the interviewer 
is added to the analysis. In the third step, the interaction between the back- 
ground of the interviewer and the background of the respondent is added 
as an independent variable. A judgement is made after every step based on 
chi-square values as to whether the explanative power of the model 
improves. 

In the first row of Table 5.2 the percentage of variance that is explained 
by the interviewers is shown. This is the variance per interviewer, so not 
only the variance as related to the ethnic background of interviewers, but 
also that related, for example, to sex, interviewer style and precision. The 
results show that most variance is explained by interviewer effect on the 
scale ‘orientation towards contacts with the Surinamese group’. By con- 
trast, the influence of the interviewer is zero on the scale “contacts with 
Surinamese”. A percentage between three and seven is found for the other 
scales. Groves (1989) states that an interviewer variance above 2 per cent 
means that interviewers have too much influence on the answers of re- 
spondents. In our research, interviewer variance below 2 per cent occurs 
only with questions on contacts with Surinam via letters and telephone. On 
all other measures the interviewer variance is above Grove’s limit. 

This means it is useful to analyse the interviewer variance further. It 
should be noted that no interviewer held more than 25 interviews, which 
is, compared to other research, a relatively small number. Because of this 
small number, the variance per interviewer can turn out to be relatively 
high. Steps one, two and three are shown in Table 5.2, indicating whether 
the model improves significantly (chi square). In the cases of significant 
improvement the un-standardised regression coefficients are shown (B co- 
efficients). Step one, adding background, sex, age, main daily activity and 
educational level of respondents, significantly improves the model with the 
five ethnic-related subjects and with education, but not with loneliness. 
This means that the background of the respondents influences the answers 
on ethnic-related subjects. 

Step two of the analysis focuses on the influence of the background of 
the interviewer. This explains part of the total interviewer variance. Adding 
the background of the interviewer to the model leads to significant im- 
provement of the model compared to step one only in the case of orienta- 
tion towards the Surinamese group. With the other subject matters, there 
was no significant improvement. The measure ‘orientation towards contact 
with the Surinamese group’ attracted our attention earlier with the highest 
interviewer effects. It is possible that it is not the background of the inter- 
viewer itself which explains the different answering patterns found be- 
tween Surinamese and Dutch interviewers, but that the background of the 
interviewer and the background of the respondent influence each other. A 
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respondent of mixed Surinamese/Dutch parentage could, for example, feel 
less distant from a Dutch interviewer than a respondent with two 
Surinamese parents. Therefore it is useful to add step three to the multi-lev- 
el model on the interaction between the background of interviewer and re- 
spondents. As the third row of Table 5.2 shows, this addition does not lead 
to significant improvement on any of the scales. 

The multi-level analysis shows that “orientation towards contacts in the 
Surinamese group’ is more susceptible to interviewer effects than any of 
the other ethnic-related subjects; actually, this is the only subject area 
where the background of the interviewer influences the answers of re- 
spondents. The subject is both ethnic-related and personal. This confirms 
our hypothesis that the strongest effect will occur with a combination of 
characteristics. Our hypothesis that influence of the background of the in- 
terviewer would also be visible with ethnic-related subjects and subjects of 
a personal nature is not confirmed. The background of respondents influen- 
ces all scale scores of ethnic-related subjects, as would also have been 
shown with a simpler analysis of variance. The background of the respond- 
ents did not matter for non-ethnic-related subjects like loneliness and edu- 
cation. The difference between the multi-level analysis and an analysis of 
variance (ANOVA) is one extra variable and two levels. The multi-level 
analysis shows that age also influences three measures, namely, orientation 
towards contacts in the Surinamese group, orientation towards the 
Surinamese language and culture and position allocation, while this was 
not shown with ANOVA. But for the influence of sex on contacts in 
Surinam the opposite is true: this was found with ANOVA but not with the 
multi-level analysis. 


5.5 Conclusion 


Our main research question asked whether results might differ depending 
on the ethnic background of the interviewer in our survey on ethnicity 
among second-generation Surinamese. I suggested that there would be an 
order in the strength of the interviewer effects from strongest for subjects 
that combine ethnic and personal issues, to less strong for ethnic-related 
subjects alone, to even weaker for personal issues and nearly zero for neu- 
tral subjects. Results based on mean scores have shown that nearly all 
measures and scales used in this study were sensitive to interviewer effects. 
This is true for the scale with ethnic-related subjects but also for education- 
al level and loneliness. The only subject matter which revealed no differen- 
ces between the two groups of interviewers was that measured by the scale 
on contacts with Surinam via post and telephone, composed of rather fac- 
tual questions. In all other instruments 3 to 11 per cent of the variance was 
explained by the ethnic background of the interviewer. The scale scores of 
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respondents with a Surinamese interviewer were systematically higher than 
the scale scores of respondents with a Dutch interviewer. This means that 
it is Important that future research takes interviewer effects into account. 

But, when we used a more sophisticated method of analysis — the multi- 
level analysis — the interviewer effects diminished. Only for one of the sev- 
en scales was the interviewer effect significant, namely the scale “orienta- 
tion towards contact with the Surinamese group”. This was the scale com- 
bining ethnic-related and personal subjects. On other ethnic-related, per- 
sonal and neutral subjects, the results were not significant. This more 
sophisticated method reveals that there is much more interaction between 
variables than can be seen using simpler methods of analysis. This means 
that the use of multi-level analysis methods should be considered in future 
research. 

Regarding explanations for the interviewer effects found in this study, it 
seems clear that the social desirability theory is supported. Pointing in this 
direction is the fact that questions such as ‘do you like to meet Surinamese 
friends’ are more frequently answered positively with a Surinamese inter- 
viewer and questions like ‘do you like to meet Dutch friends’ are more fre- 
quently answered positively with a Dutch interviewer. The question of 
which answers are ‘correct’ has not been answered. Further study of the 
actual interaction between interviewer and interviewee during the interview 
process is necessary. Particularly the way in which interviewer and re- 
spondent react to each other might provide an explanation for the inter- 
viewer variance shown in this study. Because the interview situation is al- 
ways one in which two people meet, respondents relate questions about in- 
terethnic relations directly to the relationship with the interviewer. This 
common feature explains why the interviewer effect is particularly notice- 
able regarding ethnic-related subjects. As with the subject of feminism and 
a male interviewer and female interviewees (Kane & Macaulay 1993), the 
combination of ethnic-related and personal subjects is not neutral in the in- 
terview situation. When the ethnic background of the interviewer differs 
from that of the interviewee, the interviewee will stress the similarities with 
the interviewer, or at least try to diminish differences. 

Future research must be aware of the different kinds of interviewer ef- 
fects, and it would be wise to look not only into the characteristics of inter- 
viewers but also into the nature of the interaction between interviewer and 
respondent. 


Notes 


1 Because most of the literature refers to black and white interviewers, I also use this termi- 
nology, though it combines many different ethnic groups. 
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2 Dutch municipalities define ethnic groups on the basis of birthplace and birthplace of 
parents. 

3 Surinamese populations of African, Indian, Native American (Arawaks Akurio, Trió, 
Wayarekule, Warrau and Wayana), Chinese, Jewish, Javanese and Dutch descent exist 
and other Surinamese are very often people of mixed background, so actually there is no 
general skin colour. 
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6 Surveying migrants and migrant associations 
in Stockholm 


Gunnar Myrberg 


6.1 Introduction 


This chapter presents two surveys conducted in the metropolitan region of 
Stockholm during 2004 and 2005. The first is an individual survey with a 
sample of migrants and descendants of migrants from Chile and Turkey to- 
gether with a ‘control sample’ of native Swedes. The other is a survey of 
voluntary associations organising different migrant groups from Chile and 
Turkey. Together, these two surveys form the empirical core of the 
Swedish research project Ethnic Organisation and Political Integration in 
the City. 

This chapter will pay little attention to the issue of sampling. Thanks to 
the high accuracy and completeness of Swedish population statistics, the 
technical aspects of sampling were not exceedingly problematic in our 
study. Thus, we had the opportunity to concentrate on how to achieve high 
participation rates and, equally important, high response quality in our two 
surveys. As we were not performing any kind of methodological experi- 
ments as part of our study, we are generally not able to prove that our 
choices resulted in a better study than would otherwise have been the case. 
Nevertheless, we hope that our work may serve as a source of inspiration 
for researchers who are considering similar projects. 

This chapter consists of five sections. The first provides a brief descrip- 
tion of our research project. The second, and longest section, is devoted to 
the preparatory phase of the individual survey. Here we consider issues of 
questionnaire development, translation, interview methods and other practi- 
ces with bearing on survey participation and response quality. In the third, 
we look at one aspect of the “outcome” of these preparations, namely non- 
response. In the fourth section, we present the associational survey and 
compare it to the individual survey with regard to preparations and out- 
come. In the fifth and final section, we draw conclusions on our experien- 
ces from these two surveys and discuss to what extent surveys of migrant 
populations actually require different methodological steps and measures 
from other surveys. 
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6.2 The project 


Like many European countries, Sweden today can be described as a 
‘monocultural’ nation with a few multicultural cities, at least in terms of 
demography (see Rogers, Tillie & Vertovec 2001; Borevi 2002). Currently, 
about one sixth of Stockholm’s population was born abroad and another 
tenth have at least one parent born abroad (Statistics Sweden 2004). 
Consequently, the number of inhabitants with a background from other 
parts of the world is by now large enough to permit the inclusion of fairly 
substantial subgroups of ethnic minorities in survey research. 

The overarching aim of the project Ethnic Organisation and Political 
Integration in the City was to study the role of voluntary associations as a 
political resource for a number of migrant groups in Stockholm.? There is al- 
so a strong comparative stance to the project thanks to a close collaboration 
with scholars involved in similar projects in other European cities (see chap- 
ter 7 in this book). One of the theoretical assumptions underlying this com- 
parative project is that organisational life may facilitate members’ political 
participation both by increasing their political competence and efficacy and 
by offering a channel of influence through the association itself (Myrberg 
2007; Strómblad & Bengtsson 2009). Following this assumption, a proper 
assessment of the political integration potential of voluntary associations has 
to be grounded in data collected both at the individual level (an individual 
survey) and at the associational level (an associational survey). 

Our project is focused on migrants from Chile and Turkey and their de- 
scendants. However, due to the ethnic diversity among migrants from 
Turkey, we are really studying four different migrant groups in this project: 
1) Chileans, 2) Turks, 3) Kurds with a background from the present terri- 
tory of Turkey, and 4) Syrian Christians from the same territory. 

Both Chile and Turkey are important countries of migration to Sweden 
and the Stockholm region. Moreover, a considerable number of immigrants 
from Chile and Turkey have been living in Sweden for a long time, which 
means that they have had the opportunity to create an associational life of 
some political importance (Hjarnó 1998; Mella 1990; Westin 2003). Yet, 
these groups do not seem to have reached a degree of social and political 
integration where their ethnically based associational life has lost its politi- 
cal relevance. We also know from other research that migrants from these 
two countries still face prejudice and discrimination in Swedish society 
(Lange 1995; Myrberg 2007). 


6.3 Preparing the individual survey 


In a review of existing practices to improve migrant survey participation in 
Europe, Barnes (2008: 25) concludes that the main reasons for low survey 
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participation among migrants are “language difficulties, a limited sampling 
frame, non-contact/broken appointments and refusals'. According to 
Barnes, the most widely used practices to counter these problems are trans- 
lation of survey materials — mainly the questionnaire of course, but also ad- 
vance survey letters, show cards and other information — followed by the 
use of multi-lingual interviewers and a combination of native interviewers 
and interpreters. 

We believe that both translation of survey materials and the use of inter- 
preters or multi-lingual interviewers are necessary measures in order to 
achieve acceptable levels of survey participation among migrants. 
However, this does not mean that other aspects of the preparatory phase, 
such as questionnaire development and interviewer training, can be ne- 
glected. In the following five sub-sections, we will describe how we ap- 
proached a number of steps and challenges involved in the preparation of 
the individual survey: questionnaire development; translation; interview 
method and interviewer training; pilot interviews; and the information letter 
and incentives. 


Questionnaire development 


The questionnaire used in the individual survey mainly includes well-tried 
questions that enable comparisons with traditional surveys about associa- 
tional affiliation and political participation.? There are, however, a few in- 
novations in our questionnaire that deserve to be mentioned. 

First, we had to find a way to handle the above-mentioned fact that 
migrants from Turkey belong to different ethnic groups and demand rec- 
ognition as such, not least in terms of organisation and political action. 
For instance, the Kurdish and the Syrian Christian groups are both visi- 
ble and vocal in the Swedish political arena (Deniz 2001).* However, 
there is no recording of ethnic belonging in Swedish population registers, 
which means that the only available sampling procedure in a study of 
this kind is one based on country of birth. Thus, all we knew in advance 
about the respondents of Turkish descent was that either they or their pa- 
rents were born in Turkey. For this reason, we explicitly asked the re- 
spondents belonging to this category about their ethnic identity.? Thanks 
to this question, it became possible to analyse not only differences be- 
tween migrants with Turkish descent and Chilean descent but also be- 
tween respondents defining themselves as ethnic Turks, Kurds and Syrian 
Christians. 

Second, voluntary associations organising immigrants in Sweden some- 
times carry the label “immigrant organisation” in their name. In many 
cases, however, although voluntary associations mainly organise immi- 
grants, often from the same ethnic group, the primary aim of the associa- 
tion is related to a specific kind of activity rather than to organising a 


134 GUNNAR MYRBERG 


specific group of people. This is, for instance, the case with a large num- 
ber of sports associations in Stockholm. Since one of our research ques- 
tions was to what degree people belonging to different ethnic groups in 
Stockholm affiliate with ethnic associations as compared to other kinds of 
associations, we decided to ask all respondents of Chilean and Turkish 
descent to give a rough estimate of the proportion of members with a mi- 
grant background in the associations to which they were affiliated.* 
Thanks to this question, we were able to show that in some ethnic 
groups, mainly those with a background in Turkey, affiliation with ethnic 
associations is more widespread than has been shown in earlier research 
(Myrberg 2009). 

Third, in the sections regarding political attitudes and political participa- 
tion, we asked about the ‘direction’ of the respondents’ political interest 
and activities, in terms of different political arenas.’ For example, we asked 
all respondents separate questions about their interest in local and national 
politics in Sweden and in international politics. Respondents with an immi- 
grant background were also asked questions about their interest in politics 
in Chile and Turkey. Interestingly, the analysis of these questions revealed 
that ‘native Swedes’ are somewhat more interested in local politics than re- 
spondents with migrant backgrounds, while there is no difference at all be- 
tween natives and migrants in terms of interest in national and international 
politics. We also found that the average level of interest in politics in Chile 
and Turkey among the migrant respondents was considerably lower than 
their level of interest in Swedish national politics. The only exception was 
a relatively high interest in politics in Turkey among respondents defining 
themselves as Kurds. 

Fourth, parts of the questionnaire used in the individual survey were de- 
veloped in close collaboration with our partners in the European research 
network ‘Multicultural Democracy in European Cities’. In this process, we 
had to cope with the fact that individual researchers, including ourselves, 
tend to think quite idiosyncratically about issues of migration and ethnic 
relations. To give an example, the working definitions of who is considered 
to be a migrant and what it means to belong to an ethnic group vary con- 
siderably between different European countries. This calls for a question- 
naire construction that allows both meaningful comparisons across coun- 
tries and meaningful use of the materials in different social and political 
contexts. The solution that we finally settled for was to create a number of 
detailed factual questions about the country of birth, year of migration and 
cause of migration of the respondent and of the respondent’s parents. We 
also asked about the respondents’ sense of belonging to different kinds of 
groups, including ethnic groups. These questions could then be used in dif- 
ferent combinations in different contexts, while still maintaining the poten- 
tial for cross-country comparisons. 
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Translation 


Both the questionnaires and the show cards used in the individual survey 
were translated into (Chilean) Spanish and Turkish. The main reason was 
that we felt uncertain about migrant respondents” proficiency in Swedish. 
We also had the idea that it would be more convenient, and cheaper, to 
translate the questionnaires and the show cards, than to use professional 
interpreters in every single interview with migrant respondents. As it 
turned out, a majority of the migrant respondents, including those with a 
high level of proficiency in Swedish, expressed their appreciation for the 
possibility to look at the translated questionnaires and show cards during 
the interviews, Just to make sure that they had understood the questions 
correctly. In addition, the need for professional interpreters was greatly di- 
minished. In total, only 16 respondents asked for a professional interpreter 
while another 24 interviews were conducted with the assistance of a 
relative. 

As mentioned above, the “common core” of the individual survey was 
developed in English. This meant that we first had to translate these parts 
of the questionnaire into Swedish before we could translate the entire ques- 
tionnaire into Spanish and Turkish. We conducted the translation from 
English into Swedish ourselves, and then we hired translators, trained in 
political science or sociology, with (Chilean) Spanish and Turkish as their 
respective mother tongue to complete the translation. Their translations 
were then double-checked by researchers in our team with some fluency in 
Spanish and Turkish. In the terminology of Behling and Law (2000: 16- 
24), we thus used a form of modified direct translation with medium levels 
of informativeness, source language transparency and security.* 


Interview method and interviewer training 


The questionnaire used in the individual survey was highly demanding for 
the respondent and our fear was that using either telephone or mail surveys 
would result in devastatingly low response rates (Hox & De Leeuw 1994). 
For instance, similar individual surveys in Sweden and Norway using tele- 
phone interviews have produced response rates below 20 per cent among 
some migrant groups and also surprisingly low response rates among the 
native population (Báck $ Soininen 2004; Rogstad 2007). Another argu- 
ment in favour of face-to-face interviews was that they allow the use of 
translated versions of the questionnaire and the show cards. 

The amount of training that interviewers receive has been shown to be 
critically important to the way they do their jobs (Groves, Fowler, Couper, 
Lepkowski, Singer & Tourangeau 2004; Fowler & Mangione 1990; Billiet 
& Loosveldt 1988). In our case, we had the opportunity to work with pro- 
fessional interviewers from Statistics Sweden, which meant that we could 
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focus the interviewer training on the specific challenges involved in our 
survey, rather than on basic skills. 

Before training day, the interviewers received written instructions com- 
prising about thirty pages. The first five to six pages provided a back- 
ground to the research project and some information about the different 
ethnic groups covered by the study. This was followed by about ten pages 
of general information about the survey, including the general disposition 
of the questionnaire and the construction of scales and show cards. This 
part also included the information given to the sample persons in the con- 
tact letter, together with a number of arguments that might be helpful in 
persuading the sample persons to participate. Finally, the instruction in- 
cluded ten to fifteen pages of detailed instructions about question phrasing, 
probing of inadequate answers and recording. 

The persuasive arguments probably formed the most innovative part of 
these written instructions. These arguments are also interesting since they 
reveal how we perceived the methodological challenges facing us at the 
contact stage: 


If the sample person says that he or she is not interested in politics, 
please stress the fact that the survey is not primarily about govern- 
ment issues or about political parties, but about the possibilities for 
ordinary people to improve their daily lives. 

If the sample person says that he or she is too busy, please show 
understanding but also try to insist on finding a time for an inter- 
Vlew. 

If the sample person shows concern about his or her personal integ- 
rity, please explain the concepts of confidentiality and anonymity, 
and emphasise that the protection of the respondent's personal in- 
tegrity is protected both by the scientific community’s rules of con- 
duct and by Swedish law. 


Thanks to the written instructions that we had sent to the interviewers in 
advance, and probably also to the fact that we had involved some of the in- 
terviewers at an early stage of the questionnaire development process, the 
training day went very smoothly. In fact, we were much encouraged to dis- 
cover that the interviewers seemed genuinely interested in our research 
questions and that they were highly dedicated to their task. 


Pilot interviews 


Three months before the actual fieldwork of the individual survey, eight pi- 
lot interviews were conducted by interviewers from Statistics Sweden. Two 
of these interviews were conducted in the presence of researchers from our 
research team. All eight respondents were migrants born in Chile or 
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Turkey. At this stage, however, we had not translated the questionnaire and 
the show cards into Spanish and Turkish. The pilot interviews consisted of 
two parts. First, there was the actual interview averaging about 90 minutes. 
Then, there was a follow-up interview in which the respondent and the in- 
terviewer freely discussed different aspects of the questionnaire. The latter 
conversations were tape recorded and later transcribed into a fifty-page re- 
port that turned out to be extremely valuable in the final stage of question- 
naire development. 

Three things became particularly clear during the pilot interviews. First, 
the average duration of the interviews was far too long. After about an 
hour, the respondents became tired and rapidly started to lose concentra- 
tion. Second, some of our question schemes simply did not work out in 
practice. This was most apparent in the section about participation in vol- 
untary associations where our ambition to be as conclusive as possible only 
resulted in the respondent becoming frustrated by the repetitive character 
of our questions. Third, the wording used in the questionnaire was still far 
too abstract. There was also a strong need for translations of the question- 
naire and the show cards into the respective mother tongues of the re- 
spondents. Taken together, these lessons gave us a lot of work during the 
last months before the start of the actual fieldwork. Among other things, 
this work resulted in a 40 per cent reduction of the questionnaire, including 
much simpler constructions of the question schemes concerning both asso- 
ciational affiliation and political participation. 


Information letter and incentives 


Prior to the individual survey, our research team informed the national as- 
sociations of the respective migrant groups in Sweden, including the 
Kurdish and the Syrian Christian national associations, about our project 
and received their consent for the study. The interviewers were instructed 
to inform sample persons of this consent if they were asked, but as far as 
we know that never occurred. 

All sample persons received a letter containing information about the 
purpose of our study together with information about confidentiality and 
use of data in official registers such as age, gender, income and place of 
residence. The letter was written in Swedish on one side and translated into 
either Spanish or Turkish on the other side. The letter also stated the fol- 
lowing information: 1) that each sample person had been selected purely 
by chance and could not be replaced by any other person, 2) that the sur- 
vey was financed by public research funds and that no commercial or polit- 
ical interests were involved, and 3) that each respondent would be re- 
warded with an € 11 gift voucher. Shortly after the letter had been sent, an 
interviewer contacted the selected individual by telephone to decide a time 
and place for the interview. 
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The use of economic incentives is contested in the survey methodology 
literature. We find it reasonable to offer a small compensation to the re- 
spondents for the time and energy they spend during the interview, but 
concerns have been expressed that respondents who participate in surveys 
only to reap the benefit of the incentive will increase problems of response 
bias and internal non-response (Dodd 1998). However, in their literature 
review on the issue of monetary incentives in social surveys, Simmons and 
Wilmot (2004) find little evidence to substantiate these fears. On the con- 
trary, Simmons and Wilmot (2004: 1) and others (Singer 2002; Davern, 
Rockwood, Sherrod & Campbell 2003; Goyder 1994; Hopkins & 
Gullickson 1992) argue that in general, ‘the use of incentives, however 
small in monetary terms, is effective in increasing response rates in postal, 
telephone and face-to-face surveys’. They also stress that monetary incen- 
tives have repeatedly been found to increase cooperation among certain 
groups: ‘low-income and low-education groups, larger households and 
households with dependent children, minority ethnic groups and younger 
respondents’ (Simmons & Wilmot 2004: 6). These are exactly those groups 
that survey researchers normally find it most difficult to reach. 


6.4 Non-response in the individual survey 


The population of the individual survey was defined as all residents in the 
region of Greater Stockholm, aged 18-74 years, and belonging to one of 
the following three categories: individuals born in Chile and individuals 
born in Sweden with at least one parent born in Chile (Chilean descent); 
individuals born in Turkey and individuals born in Sweden with at least 
one parent born in Turkey (Turkish descent); individuals born in Sweden 
with both parents born in Sweden (Swedish descent).? 

Within each category, the central government authority for official statis- 
tics and government statistics, Statistics Sweden, was assigned to conduct 
a random sampling procedure with gender stratification based on continu- 
ously updated register data. The gross sample size was 500 individuals in 
each category. Due to an over-coverage of 47 individuals, the total net 
sample size was 1,453 individuals. "° 

As we see in Table 6.1, the proportions between the sample and the pop- 
ulation differ markedly between the three main categories. For example, 
our sample of people of Chilean descent in the region of Stockholm ac- 
tually corresponds to about 4 per cent of the entire population. This fact 
has at least two important consequences. First, a survey of this kind will 
be known rapidly within the entire migrant community. If the survey is 
perceived as interesting and fair, this may turn out to be an advantage. If, 
however, the first respondents dislike the survey for some reason, the like- 
lihood of achieving a good response rate plummets. Second, and on a 
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Table 6.1 Population and sample in the individual survey 


Turkish descent Chilean descent Swedish descent Total 


Men Women Men Women Men Women 


Gross sample (N) 260 240 255 245 250 250 1,500 
Population (N) 9,180 8426 6,012 5,899 355,187 354,257 738,961 
Gross sample/population (%) 2.8 2.8 4.2 4.2 0.07 0.07 

Net sample (N), excluding 254 231 243 235 246 244 1,453 


over-coverage 


Source: Author's own elaboration, based on Ethnic Organisation and Political Integration in 
the City (individual survey) 


different note, the high probability of migrants being selected in random 
samples of surveys of migrants, in combination with the increasing number 
of these surveys, is likely to cause serious problems of survey fatigue 
among migrants in the years to come. In other words, the rapidly growing 
“industry” of migrant surveys in Europe is facing a collective action dilem- 
ma, which it has yet to address in a structured manner. 

Returning to our survey, the fieldwork started in February 2004 and 
ended in October the same year. The survey was carried out as computer- 
assisted face-to-face interviews averaging about 55 minutes in length. Most 
of the interviews took place in respondents” homes. However, about one 
third of the interviews were carried out in other places, such as libraries, 
restaurants and cafés. In general, this had to do with people saying that it 
was more convenient for them to meet the interviewer near their workplace 
than at home. 

The initial fieldwork reports were very positive. Sample persons rarely re- 
fused to participate and the interviews worked much better with the final ver- 
sion of the questionnaire than with the pilot version. Gradually, however, a 
disturbing pattern emerged in terms of response rates. As it turned out, the 
interviewers were unable to even locate a large number of the sample per- 
sons of migrant background, particularly those with a background in Turkey. 

After three months, the response rates were 60 per cent among native 
Swedes, 42 per cent among people with a background in Chile and only 
32 per cent among people with a background in Turkey. At this point, 
Statistics Sweden informed us that we would have to make a strategic deci- 
sion about the direction of the fieldwork in the months following the 
summer break. Either the interviewers should be instructed to maximise 
the average response rate, which in practice would mean that they would 
concentrate on the “easy targets” (1.e. native Swedes). Or they should be in- 
structed to maximise the response rates among people with a background 
in Chile and Turkey. Since we were beginning to feel rather uneasy about 
the low response rates among the migrants and their descendants, we chose 
the latter strategy. Following this decision, all interviewers involved in the 
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survey were instructed to focus exclusively on sample persons with a back- 
ground in Chile and Turkey during the last two months of the fieldwork. 
As a result, the response rates increased by 15 and 17 percentage points, 
respectively, in these two categories during the final two months of field- 
work, compared to only 6 percentage points among native Swedes. 

In total, 838 interviews were completed, producing an overall response 
rate of 57.7 per cent. An analysis of the response figures reveals consider- 
able variation between the three sample categories: 48.9 per cent among 
the Turks, 57.9 per cent among the Chileans, and 66.1 per cent among the 
native Swedes. Importantly, however, partial non-response levels were con- 
sistently low across all three categories and throughout the questionnaire. 

A closer look at the non-response in Table 6.2 reveals some interesting 
facts. First of all, we see a stable gender pattern across all three categories. 
In each category, we find a gender gap of about 5 percentage points, with 
women constantly showing higher response rates than men. Interestingly, 
however, this gender gap seems to be for somewhat different reasons 
among migrants than among native Swedes. The main difference between 
men and women of Turkish and Chilean descent has to do with locating 
the respondent. For example, about one fourth of the male sample in these 
two categories was never located by the interviewers, despite the fact that 
the interviewers’ contact information was collected straight from the regis- 
ters of Statistics Sweden. In contrast, the gender gap among native Swedes 
has to do with men being less cooperative than women. Native Swedish 
men are simply more likely to say that they cannot be bothered or that they 
never participate in surveys than native Swedish women. 


Table 6.2 A closer look at non-response (percentages) 


Turkish descent Chilean descent Swedish descent 


Men Women Men Women Men Women 


Not capable due to disease or 1.5 3.5 2.5 2.6 1.2 0.8 
mental handicap 

Language difficulties 0.0 1.3 0.0 0.4 0.0 0.0 
Unknown address/no telephone 13.4 10.0 14.4 12.3 3.7 3.7 
Not found for other reasons 8.7 7.4 9.5 7.7 4.1 5.3 
Too busy 9.8 6.1 4.9 3.4 7.3 6.6 
Cannot be bothered/voluntariness 14.1 15.6 9.0 8.5 13.8 9.8 
Never participates in surveys 2.3 1.7 2.1 1.3 2.4 0.8 
Disliked survey matter 0.8 0.4 0.0 0.4 0.8 0.4 
Concerns about confidentiality 0.4 0.0 0.0 0.0 0.0 0.8 
Refused to participate for 2.5 2.5 2.5 2.5 3.3 2.9 
other reasons 

Total non-response 53.5 48.5 44.9 39.1 36.6 31.1 
Completed interview 46.5 51.5 55.1 60.9 63.4 68.9 
Total 100.0 100.0 100.0 100.0 100.0 100.0 


Source: Ethnic Organisation and Political Integration in the City (individual survey) 
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Secondly, it is encouraging to see that almost nobody refused to partici- 
pate in the survey due to concerns about confidentiality or because they 
disliked the survey subject matter. Of course, these reasons may form part 
of saying that one cannot be bothered or saying that one refuses for other 
reasons. Yet, we are happy to see that our efforts in presenting the survey 
and explaining the issues of confidentiality as clearly as possible seem to 
have been effective. 

Thirdly, and on the more negative side, it is difficult to see what can be 
done about the remaining reasons for non-response. If information and in- 
centives, in combination with a highly accurate sampling frame, is not 
enough to attract more than about half of the sample among certain mi- 
grant groups, we need to ask ourselves what more could possibly be done 
to raise response rates in these categories. 


6.5 The associational survey 


Turning to the associational survey, we first want to stress the similarities 
in terms of preparations and fieldwork between this survey and the individ- 
ual survey. For example, we used professional interviewers from Statistics 
Sweden in both surveys. We also used the same procedures for translation, 
although we added a language to the associational survey. We also put the 
same effort into developing the questionnaire and preparing the inter- 
viewers in order to achieve as high a response rate and response quality as 
possible in both surveys. 

This being said, there are of course also differences between the two 
studies. In the individual survey, we decided to exclude the relatively pe- 
ripheral municipalities of Norrtálje, Nykvarn, Nynáshamn and Sódertálje in 
order to reduce travelling time and expenses for the interviewers. This 
turned out to be a mistake, however, since a fairly sizeable portion of mi- 
grants with Turkish descent in the Stockholm region actually lived in the 
municipality of Södertälje, particularly people defining themselves as 
Syrian Christians. When preparing the associational survey, we thus had to 
decide whether we should repeat this mistake, and probably lose several 
Syrian Christian associations from the population, or if we should include 
the four peripheral municipalities, although this would lead to slightly dif- 
ferent geographical demarcations between the individual and the associa- 
tional surveys. In this situation, which we admittedly had put ourselves in, 
we chose the latter alternative in order to reach as many associations in the 
survey as possible. Thus, the population of the associational survey was 
theoretically defined as all associations in the County of Stockholm, in- 
cluding the four peripheral municipalities, organising migrants and de- 
scendants of migrants from Chile and Turkey. 
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In terms of content, the questionnaire used in the associational survey 
was less conventional than the one used in the individual survey. The aim 
of the associational survey was to get a picture of the internal organisation 
of associations organising immigrants and of their external contacts with 
authorities and other kinds of associations. The most sensitive issue that 
we had to deal with was how to ask about the economic situation of the as- 
sociations. This information was clearly important to our study, not least in 
the comparative perspective. However, we were afraid that some of these 
questions would cause suspicion on the part of the representatives and that 
some of them would even end the interview 1f they started to think that we 
were performing an audit of their association. For this reason, all questions 
about the economic situation were placed in the final section of the ques- 
tionnaire. We also took great care in phrasing these questions in as unpro- 
voking, yet informative, terms as possible.'! 

Another challenge in the associational study was how to feasibly map 
the external contacts of the associations in a way that would capture their 
range as well as their intensity. Our solution here was to divide different 
possible kinds of contacts into different sections of the questionnaire, thus 
dividing what was in fact one extremely ambitious question scheme into a 
number of smaller and therefore more easily digested schemes. 

In the absence of reliable register data about voluntary associations in 
Stockholm, our research team expended great effort to construct as com- 
plete a population as possible. Associations were located through a variety 
of sources, such as documentary analysis, interviews with experts and in- 
ternet searches.'* Most importantly, the interviews were preceded by a pe- 
riod of three months when associations were tracked down by research as- 
sistants with backgrounds in Chile and Turkey. During this phase, the asso- 
ciations that we located were asked to provide the name and address of a 
representative whom the interviewers could contact later. This representa- 
tive was often, but not always, the chairperson of the association. In the 
end, this work produced a list of 141 associations, including the names of 
a representative of each association together with an address and a tele- 
phone number for the association or for the representative.!? 

The associational survey was preceded by five pilot interviews ranging 
in length from 54 to 72 minutes. These interviews did not include the 
same kind of follow-up conversation as in the individual survey, but judg- 
ing from the comments we received from the interviewers the pilot ques- 
tionnaire was working much better in this survey than in the individual 
survey. 

The fieldwork for the associational survey was carried out in the autumn 
of 2005. This time, the face-to-face interviews averaged about 75 minutes. 
Questionnaires and show cards were available in Swedish, Spanish, 
Turkish and Kurdish. The translation procedure was the same as in the in- 
dividual survey, only this time we also performed translations into 
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Kurdish, since we had been warned that presenting questionnaires in 
Turkish to representatives of Kurdish associations could be problematic. 

In total, 106 interviews were completed, which represents a response 
rate of 87 per cent. Needless to say, this response rate clearly exceeded our 
expectations.! Not even the fact that the representatives were asked to pro- 
vide written material about the associations in connection with the inter- 
views seemed to deter them from participation.'* 


6.6 Concluding remarks 


Do surveys of migrant populations and migrant associations require differ- 
ent methodological measures than other surveys? The best answer to this 
question is both ‘yes’ and ‘no’: yes, in the sense that there are some instru- 
ments in the methodological toolbox that become particularly important in 
surveys of migrants; no, in the sense that many of the instruments that are 
important in surveys of migrants are also important, although sometimes 
neglected, in other kinds of surveys. 

To give an example, translation of information letters, questionnaires 
and show cards is obviously an important measure in surveys of migrants. 
Translating the information letter into the languages spoken by sample per- 
sons signals the importance of their participation in the study. Translating 
the questionnaire and show cards also carries this signal but, even more 
importantly, this measure facilitates the interview situation both for the in- 
terviewer and for the respondent. It is also a much cheaper and more con- 
venient way of solving the language problem than hiring professional inter- 
preters for each interview. Thus, translation is definitely important in sur- 
veys of migrants. Given that most countries in the world are becoming 
increasingly multicultural and multilingual, the issue of translation is 
bound to become more and more important in surveys with purely repre- 
sentative samples, not to speak of the fact that social surveys today are in- 
creasingly being designed with the explicit aim of cross-national 
comparison. 

Another example is the importance of a continuous dialogue between re- 
searchers and professional interviewers. In our study, we had the opportu- 
nity of engaging Statistics Sweden to conduct the fieldwork. This included 
meetings with an advisory board at Statistics Sweden, help with inter- 
viewer training and pilot interviews, administration, coding, data delivery 
and technical reports, which, all in all, greatly facilitated our work. Of 
course, we as researchers need to be very clear in such a dialogue that we 
are the ones making the scientifically relevant decisions. This being clear, 
however, there is much to learn from professional interviewers regarding 
the nuts and bolts of the actual interview situation. If the interviewers say 
that a certain question or a certain question scheme simply doesn’t work, 


144 GUNNAR MYRBERG 


our experience is that it is much more productive in the long run to make 
changes in the questionnaire than to stubbornly hold on to the original for- 
mulations. Of course, this is not always possible in cross-national surveys, 
but this means that one should probably include professional interviewers 
in the initial stage of such cooperation. 

While we obviously did commit some mistakes during the preparations 
for our two surveys — particularly the geographical demarcation of the re- 
gion of Greater Stockholm in the individual survey — we do feel that we 
did the best we could in the methodological design of this study and we 
would basically follow the same lines if we were to conduct a follow-up 
study. 


Notes 


1 This project was funded by the Bank of Sweden Tercentenary Foundation and by the 
Swedish Council for Working Life and Social Research. Principal investigators were Bo 
Bengtsson and Nils Hertting at the Institute for Housing and Urban Research, Uppsala 
University, and Gunnar Myrberg at the Department of Government, Uppsala University. 

2 In the individual survey, the region of Greater Stockholm was defined as the County of 
Stockholm, except for the four relatively peripheral municipalities of Norrtálje, Nykvarn, 
Nynäshamn and Södertälje. This definition of Greater Stockholm is used by the municipal 
agency for research and statistics in Stockholm (Stockholm Office of Research and 
Statistics 2003). The total population in the region of Greater Stockholm is about 1.7 mil- 
lion, of which 18.5 per cent are immigrants (Statistics Sweden 2004). 

3 The questionnaire comprises eight sections in total. The first section regards the demo- 
graphic background of the respondent including country of birth and migration history. 
This section also includes questions about the respondent’s family and social life. The 
second section is about affiliation with voluntary associations. This section is dominated 
by a rather lengthy question scheme including twenty different types of associations. The 
next two sections deal with political attitudes and political participation; the latter section 
is dominated by a question scheme about participation in twenty different kinds of politi- 
cal activities. The remaining three sections include questions about housing, employment 
and education, and media consumption. Finally, there is a brief section directed at the in- 
terviewer which includes questions about the language proficiency of the respondent. 

4 The Syrian Christian group in Sweden is, in turn, divided into two subgroups: ‘Assyrians’ 
and ‘Syrians’. The distinction between these two subgroups is based on rather opaque 
grounds, and its importance seems to vary between different spheres (Deniz 2001). Its 
most visible expression in Sweden is in the names of several successful football clubs, 
most notably the two Södertälje teams ‘Assyriska’ and ‘Syrianska’. 

5 ‘Which of the following national or ethnic identities do you identify yourself with the 
most? 1. Assyrian identity, 2. Kurdish identity, 3. Syrian identity, 4. Turkish identity, 5. 
None of these identities.’ 

6 ‘Speaking of [association(s) in which respondent is member], would you say that a major- 
ity of the members in this association are migrants?’ 

7 “You mentioned that you had participated in [name of political activity]. Which of the fol- 
lowing categories of people would you say that this activity primarily concerns? 1. 
Myself, my family or a few other people, 2. People in my neighbourhood, 3. People in 
the region of Greater Stockholm, 4. People in Sweden, 5. People in [Chile/Turkey], 6. 
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People in one or more other countries, 7. People in the whole world.” This question is an 
adapted version of a question developed by the Swedish Government Commission 
Inquiry on the Political Integration of Immigrants, which was run by colleagues of ours at 
the Department of Government at Uppsala University. 

8 Behling and Law (2000: 17) define informativeness as ‘the degree to which the technique 
produces the researcher with objective indications of the semantic equivalence of the tar- 
get language version of the instrument and pinpoints the nature of the specific problems 
with it’. Source language transparency is defined as “the degree to which the technique 
produces useful information to the researcher who lacks fluency in the target language”. 
Finally, security is defined as “the degree to which the technique builds in opportunities 
to check the work of the original translator”. 

9 In the text, we sometimes use the looser formulations ‘Chileans’, ‘Turks’ and “native 
Swedes” to denote these three categories. However, the definition of the categories is the 
same throughout. 

10 The over-coverage consists of individuals who had moved permanently from the sample 
region before the fieldwork started or who had died. 

11 “What is the size of your annual budget?” ‘Could you please indicate which of the follow- 
ing are the main posts of your budget and their size? 1. Income from sales and services, 
2. Income from events, 3. Membership fees, 4. Private contributions, 5. Contributions 
from companies, 6. Contributions from other associations, 7. Contributions from the mu- 
nicipality, 8. Contributions from the state, 9. Contributions from the European Union, 10. 
Contributions from [Chile/Turkey].” 

12 This approach was inspired by the associational surveys performed in Aalborg, Aberdeen, 
Bern, Enschede, Mannheim and Sabadell by the so called CID-network (Citizenship, 
Involvement and Democracy) funded by the European Science Foundation (see Maloney 
& Rossteutscher 2007). 

13 The population consisted of 47 Turkish associations, 42 Kurdish associations, 21 Syrian 
Christian associations (of which 10 defined themselves as ‘Assyrian’ while the remaining 
11 defined themselves as ‘Syrian’) and finally 31 Chilean associations. 

14 As a point of reference, the above-mentioned CID-surveys achieved response rates rang- 
ing from 32 per cent in Sabadell to 55 per cent in Bern. However, these surveys had a 
much larger scope and therefore also much larger populations. For example, the actual 
number of completed interviews in the Mannheim study was 1,618 (Font, Geurts, 
Maloney & Berton 2007: 23). 

15 These materials mainly included statutes and descriptions of the associations” activities. 
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7 Comparing the response rates of 
autochthonous and migrant populations 
in nominal sampling surveys: 

The LOCALMULTIDEM study in Madrid' 


Laura Morales and Virginia Ros 


7.1 Introduction 


Immigration flows have continued or intensified in the last two decades in 
many West European countries, some of which have been attracting large 
numbers of immigrants since the 1950s and 1960s. Countries that were 
previously net senders of emigrants — such as Ireland, Italy, Portugal and 
Spain — have become, since the 1980s and 1990s, net receivers. Together 
with Ireland, nowhere has this reversal of population flows changed so dra- 
matically and rapidly as in Spain (see chapter 3 in this volume). 

The strong and sustained growth of the Spanish economy during the first 
half of the 2000s, together with its ageing population structure, has fav- 
oured the inflow of migrant workers mostly from Eastern Europe, Latin 
America and the Maghreb — resulting in a steep surge, especially since the 
late 1990s. Moreover, the spatial distribution of this immigrant population 
was initially concentrated in a limited number of areas and regions in 
Spain, but in the 2000s considerable immigrant populations spread all over 
the country, even if EU immigration tends to concentrate in very specific 
areas — mainly coastal and islands.? 

In the context of such massive and rapid social change, it has become 
increasingly important in recent years to be able to adapt survey sampling 
methods and strategies so as to get an accurate representation of the opin- 
ions, attitudes and behaviours of the various groups of newcomers to the 
country (for more details, see chapter 9 in this volume).? What is lacking 
is a systematic integration of immigrants in the daily and ordinary surveys 
that are conducted in Spain; in other words, a ‘normalisation’ of their in- 
clusion in the typical survey, regardless of whether they have naturalised 
as Spanish citizens. 

Often, underlying this exclusion of non-naturalised immigrants is the 
feeling or sense on the part of opinion polling institutes and their fieldwork 
organisations in Spain that it is very difficult and costly to reach this target 
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population, and that response rates are poor for this subgroup. 
Furthermore, as we will review in the following section, some of the exist- 
ing survey research scholarship reinforces this view that the response rates 
of immigrants and individuals of migrant-background are substantially 
lower than those of the autochthonous population. In this chapter, we ex- 
amine how much truth there is to these widely held beliefs for the Spanish 
case, and we analyse the main elements that impinge on the final response 
rate of a sample of autochthonous and migrant populations in Madrid in 
the context of a survey conducted within a comparative European project 
that aimed at studying the social capital and participation levels of various 
groups of immigrants across European cities: the LOCALMULTIDEM 
project. 

One of the main challenges of the LOCALMULTIDEM project was to 
design a set of common methodological guidelines that would allow for 
the conducting of comparable surveys across a number of different 
European cities. One first obstacle to that task was related to the very dif- 
ferent nature of the available sampling frames — or the lack of appropriate 
sampling frames altogether. The second hurdle involved the substantial dif- 
ferences in the composition of the immigrant-origin population. Though 
the solutions are never fully satisfactory, the project approached both diffi- 
culties by aiming at producing the best possible random probability sam- 
ples with a selection of “functionally equivalent” groups. 

Thus, in each of the cities studied, we conducted surveys of representa- 
tive samples of residents, stratified by national/ethnic origin. In each city, 
the surveys included at least two — and in most cases three — groups of im- 
migrant origin, and a control group of autochthonous population; and we 
aimed at obtaining sub-samples for each of the groups of between 200 and 
300 individuals.* Moreover, in our selection of immigrant-origin groups in 
each city, we balanced a number of aspects that enhanced comparability: 
their population size needed to be large enough to allow for the extraction 
of a sample of 300 individuals and they had to be “relevant” in each of our 
cities; we included groups of more distant and more recent migration 
waves; we included at least one group of predominant Muslim faith; and 
we aimed at maximising the comparability of national origins across cities 
whenever possible. 

As having identical administration methods and sampling frames was 
not a feasible option, we opted for the solution prescribed by the European 
Social Survey (ESS): choosing the best practice available in each place. In 
Barcelona, Budapest, London, Madrid, Milan and Stockholm (see chapter 
6 in this volume) the interviews were conducted face-to-face, whereas — 
due to cost issues or sampling frame availability — in Geneva, Lyon, Oslo 
and Zurich they were undertaken by telephone. The sampling strategies 
had to adapt to the different availability of registers or lists that covered 
the population of interest. Hence, in Barcelona, Budapest, Geneva, Madrid, 
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Oslo, Stockholm and Zurich nominal individual samples were randomly 
drawn from the local population registers. In London, focused enumeration 
within postal districts was employed (see chapter 2 in this volume for a de- 
scription of this procedure). In Milan, a method of random selection within 
centres of aggregation was employed for the migrant groups, while the au- 
tochthonous group was selected from telephone registers. Finally, in Lyon, 
the lack of any available register that includes information about the coun- 
try of birth or nationality of the individual or on that of the parents led to a 
sample design that proceeded by randomly generating telephone numbers 
— within the area code — and screening respondents through a short list of 
questions about their country of birth and their ancestry. 

Given this variation in the methods, in this chapter we have opted to fo- 
cus on just one of the surveys, to illustrate the main issues related to the re- 
sponse rates of immigrant-origin populations. We believe that our study is 
of particular interest because there is limited information about response 
rates of immigrant populations in “new destination” countries and because 
a large share of the immigrant population shares Spanish as their mother 
tongue while many others do not. Hence, this allows for a comparison of 
the response rates of different immigrant groups of relatively recent arrival 
with different language barriers. 

The LOCALMULTIDEM survey was undertaken in Madrid in 2006- 
2007 with around 1,200 individuals stratified by country of origin. As we 
will explain in detail in the next sections of this chapter, the original sam- 
ple had been designed to be nominal, with 2,400 individuals randomly ex- 
tracted from the municipal population register, and the study aimed at em- 
ploying the same methodology as the ESS, compiling contact forms that 
are almost identical to those used by the ESS team in Spain. With the in- 
formation stemming from these contact forms, we analyse in detail the var- 
ious challenges faced in achieving high response rates from migrant popu- 
lations, and we provide comparative elements with the ESS in Spain. In 
particular, we focus on the problems of participation refusal, non-contact, 
and relocation of the sampled individual, and the extent to which they af- 
fect autochthonous and migrant populations differently. 

The chapter is structured as follows. First we review the scant literature 
that addresses the issue of response rates for immigrant and ethnic minor- 
ities in established democracies, and point out the main gaps in our knowl- 
edge about this aspect. We then describe in detail the context of the study 
and the methods of sampling and data collection for the survey. The next 
section compares the response rates of immigrant-background and autoch- 
thonous individuals in the LOCALMULTIDEM survey and in the Spanish 
ESS. The final empirical section examines how successful the strategies 
are for refusal conversion and repeated location attempts for this group 
when compared to the autochthonous Spanish. We conclude the chapter 
with some general thoughts about the most effective and efficient way to 
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approach the study of immigrant populations in Spain, and reflect on how 
applicable these findings may be in other countries. 


7.2 Response rates in survey research: What do we know about 
interviewing immigrants? 


Achieving a high response rate has been a common target for most quality 
surveys, as there is a general consensus that this reduces the likelihood of 
obtaining biased sampling results and estimates (Bradburn 1985) The 
general recommendation is that random probability sampling at all stages 
of the unit selection process, avoiding substitution and pursuing a high re- 
sponse rate are the best ways to achieve a sample that is truly representa- 
tive of the target population (Fowler 2002; Stoop 2005). In Europe, the 
European Social Survey (ESS) has become a quality standard in social sur- 
vey research by requiring highly demanding methodological practices of 
all the participating countries (see Stoop, Billiet, Koch & Fitzgerald 2010 
for a detailed description). Among other strategies, the ESS seeks to en- 
hance response rates by: 1) setting a minimum sample size of 2,000 indi- 
viduals and an effective sample size of 1,500;° 2) asking participating 
countries to aim at a response rate of at least 70 per cent; and 3) requiring 
that four visits or contact attempts be made before giving up on any sample 
individual or household, at least one of which should take place during the 
late afternoon or evening and one during a weekend. Moreover, putting in 
place conversion refusal and location strategies is strongly encouraged by 
the central coordinating team of the ESS; and all contact attempts and their 
results are duly registered in standardised contact forms that are used in all 
participating countries (see Billiet, Philippens, Fitzgerald & Stoop 2007 for 
more details). 

As Stoop et al. (2010) report, the goal of achieving a high response rate 
that reaches at least 70 per cent has resulted in varying success across 
countries and over time. In the first round of the ESS (2002-2003) only 5 
of the 25 participating countries were able to meet this target, and in the 
fourth round of the ESS (2008-2009) 7 of 28 achieved it; though many 
participating countries have been able to gradually improve their response 
results, as has the Spanish team of the ESS, which has implemented vari- 
ous strategies to improve response rates (see Riba, Torcal & Morales 
2010). This has resulted in an increase from the initial response rate of 
53.2 per cent in the first round, to the 66.8 per cent achieved in the fourth 
round.’ The Spanish results are especially important for our own study as 
it is the benchmark against which we can evaluate the quality and success 
of our own survey of individuals of immigrant origin in Madrid. 

However, an important limitation of the ESS and other general-purpose 
surveys is that they include a very small number of persons of immigrant 
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origin or include none at all. In fact, for the fourth round, the technical re- 
ports for at least two countries — Israel and France — acknowledge that for- 
eigners (i.e. non-citizens) are not included in the sampling frame, while the 
information is not sufficiently clear for Estonia, Finland and Poland. 
Moreover, most ESS participating countries that use a population register 
as their main sampling frame, even if the register includes resident foreign- 
ers, exclude unauthorised immigrants, with Spain being a notable excep- 
tion (see chapter 3 in this volume for more details). Hence, for some coun- 
tries, this additional group of the resident immigrant population is not rep- 
resented in the ESS survey.* Altogether, this limits our capacity to learn 
much about different inclinations to respond among immigrants from one 
of the major social surveys in Europe, and it is more appropriate to turn 
our attention to surveys that are specifically designed to cover populations 
of immigrant origin. 

However, a first limitation when framing the study of the survey re- 
sponse rates of individuals of immigrant origin is the limited availability of 
research that focuses primarily on this subgroup of the population. Often, 
the analysis of immigrants? survey response rates is confounded with the 
lower response rates of urban dwellers, thus, the evidence is inconclusive 
(see Stoop et al. 2010: 125-126). On other occasions, results of surveys of 
immigrants and ethnic minorities are treated as 1f they were strictly equiva- 
lent. Beyond the fact that some ethnic minorities are not part of (relatively 
recent) migration flows, but may have been members of the given society 
for centuries, key differences between first and subsequent generations of 
individuals of immigrant background are present in regards to their country 
of socialisation. In fact, this differential socialisation across generations of 
individuals of immigrant background is probably the origin of the reversal 
in survey response inclinations among ethnic minorities reported in 
Feskens, Hox, Lensvelt-Mulders and Schmeets (2006) in comparison to 
Bronner’s (1998) earlier diagnosis. 

For this reason, it is better to focus on the analyses of fieldwork efforts 
and results that pertain specifically to surveys of immigrants and their im- 
mediate descendants. In this regard, one interesting finding from reviewing 
the few existing studies is that, in certain contexts, immigrants show over- 
all higher response rates than the respective autochthonous or native popu- 
lation, while they are less likely to be covered in other contexts. For exam- 
ple, Blohm and Diehl (2001) show that when appropriate and adapted 
fieldwork operations are put in place — such as bilingual interviewers and 
the possibility to respond in the native language — response rates from im- 
migrants can be 20 percentage points higher than for the overall popula- 
tion.” In contrast, Deding, Fridberg and Jakobsen (2008 and chapter 8 in 
this volume) show that, even with interviewing available in their native 
language, the three immigrant groups they study — from Iran, Pakistan and 
Turkey — have response rates 20 percentage points lower. 
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Response rates are, fundamentally, determined by the capacity to locate 
individuals at the address or place related to sampling, their ability to re- 
spond to the survey — sometimes limited by illness, disability or language 
proficiency — and their eventual cooperation with interviewers when con- 
tacted. Locating immigrants is the initial obstacle in many societies, as im- 
migrants are commonly a younger, less stable population, and often work 
longer hours. As Feskens et al. (2006) show, non-contact rates are higher 
for immigrant-origin individuals in several European countries, and these 
substantially lower contact rates hold when socio-economic status, urban- 
isation and several other demographics are controlled for (Feskens, Hox, 
Lensvelt-Mulders & Schmeets 2007). ° 

Furthermore, Blohm and Diehl (2001), in analysing the nationwide 
German ALLBUS survey, report that wrong or false addresses are quite a 
problem with this sector of the population. This is the source of a large part 
of the non-locations in the survey. But these authors note that the problem 
of tracking immigrants who have moved elsewhere can be overcome with 
appropriate fieldwork strategies. Also, it is not clear that locating immi- 
grants is significantly more difficult than for the majority population — oth- 
er than in the case of wrong or false addresses. 

An interesting result that Blohm and Diehl (2001) uncover is that certain 
groups within the immigrant population can be particularly difficult to con- 
tact; in their case, older immigrants and men. Similarly, Deding et al. 
(2008) underscore that immigrants of certain national origins are much 
more difficult to locate than others — in Denmark, Pakistanis even after 
controlling for socio-demographics. Hence, blanket statements about, and 
response-enhancement strategies relating to, ‘all’ immigrants can be poten- 
tially misleading. Additionally, Feskens et al. (2007) show that certain so- 
cio-demographic characteristics — such as age — are differently related to 
non-contact likelihoods in immigrant and native populations. 

With regard to language barriers, which are so commonly a source of 
non-response in surveys with immigrants and ethnic minorities (Feskens et 
al. 2006), several of these studies show that with the appropriate fieldwork 
practices in place — bilingual questionnaire and bilingual interviewers — their 
impact can be reduced to a minimum; although they are still potentially 
problematic with immigrants originating from multi-lingual countries 
(Blohm & Diehl 2001; Deding et al. 2008). However, one possible problem 
with offering multi-lingual interviews is that differential language adminis- 
tration can introduce measurement errors related to translation, cultural 
framing and situational cues (Peytcheva 2008a, 2008b). These are in addi- 
tion to the difficulties introduced in the process of supervising the work of 
the interviewers. 

The evidence regarding whether immigrants are more or less cooperative 
than the rest of the population is mixed (Feskens et al. 2006). Blohm & 
Diehl (2001) show that immigrants have lower refusal rates both in their 
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own local survey of immigrants and in the nationwide ALLBUS survey; 
but Feskens et al. (2007, 2008) suggest that this might be limited to immi- 
grants of certain origins. They find differing cooperation patterns between 
Western and non-Western immigrants in the Netherlands. !! In contrast, 
Deding et al. (2008) find that cooperation is lower — sometimes substan- 
tially so — for immigrants (in their case all non-Western groups) than for 
native Danes. Similar results of higher refusal rates among immigrants in 
the United Kingdom are provided by Thomas (2008). 

Additionally, 1t is not clear that monetary incentives necessarily increase 
cooperation rates from immigrants even if they seem to be successful with 
native respondents, partly because baseline cooperation rates for these 
groups are already higher in some countries (Van den Brakel, Vis- 
Visschers & Schmeets 2006; Feskens et al. 2008). There is limited infor- 
mation in previous studies about the success of refusal conversion at- 
tempts. However, Blohm and Diehl (2001) find that the geographical prox- 
imity of interviewers to the location of respondents” addresses increases 
the inclination to cooperate, which is probably indicative of the success of 
(some) such strategies. 

Finally, with respect to the effect of interviewer characteristics on suc- 
cess rates, the evidence is also mixed. Blohm and Diehl (2001) find that 
they matter for the case of Turks in Mannheim and that — unlike common 
preconceptions — male interviewers yield higher response rates from female 
respondents. In contrast, Deding et al. (2008) find no evidence of inter- 
viewer effects related to their socio-demographics, but only an effect re- 
lated to interviewing experience. 

In summary, an examination of the existing studies on non-response in 
surveys of individuals of immigrant origin suggests a number of elements 
that are worth considering in this chapter. First, we have found no previous 
study of immigrants” response rates in “new destination” countries — such 
as South European countries. The Spanish case is of particular interest be- 
cause a large share of immigrants to Spain are of Latin American origin 
and share their mother-tongue with the autochthonous population, while 
other immigrant groups do not. This permits a comparison on the various 
issues pertaining to response rates in different subpopulations. Second, a 
common conclusion of all studies is that urbanisation is a confounding var- 
table with immigrant status, hence the fact that we are analysing the sub- 
ject in the largest city in Spain substantially reduces this problem, as both 
the native and the immigrant groups reside in the same highly urbanised 
city. Third, some of the studies suggest that contact and cooperation rates 
might vary across immigrant origin groups. In this regard, our systematic 
comparison of five different national origin immigrant groups — one 
Maghrebian and four Latin American — allows us to explore this further. 
Finally, our comparison of the LOCALMULTIDEM survey with the ESS 
in Spain enables us to ascertain whether using equivalent methods for 
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locating respondents and for converting refusals is similarly successful 
among immigrant populations. 


7.3 The context and the study 
The immigrant population in Madrid: An overview 


Along with high rates of undocumented status, a major characteristic of the 
immigrant population in Spain is its uneven geographical distribution 
throughout the country. In 1996, approximately 53 per cent of the total for- 
eign population in Spain resided in four out of the country’s seventeen re- 
gions: Andalusia (16 per cent), Catalonia (16 per cent), Valencia (14 per 
cent) and Madrid (17 per cent). In 2005, these four regions concentrated 
about 67 per cent of the total foreign-born population living in Spain. 
These figures provide an idea of the extent to which immigration is a rela- 
tively geographically delimited issue. Madrid, as the capital city, has at- 
tracted a large percentage of the total immigration to Spain. In 1996, the 
immigrant population living in the city of Madrid represented approxi- 
mately 10 per cent of the total foreign-born residents in Spain; ten years 
later, the corresponding percentage had increased to 12.5 per cent. Even if 
the immigrant population living in the city of Madrid is not necessarily 
representative of the overall immigrant population, many of its features are 
also applicable to the description of the immigrant population living in oth- 
er large Spanish cities. 

Foreign-born individuals represented approximately 3.7 per cent of the 
total population of Madrid in 1996;!2 in ten years this percentage increased 
to 18 per cent. In other words, the proportion of the immigrant population 
in the city increased from less than one-twentieth to almost one-fifth of the 
total population over just one decade. In addition, such substantial growth 
has not been evenly distributed over this time period, but occurred primar- 
ily after 2000 (Figure 7.1) 

Hence, the immigrant population in the city of Madrid has increased 
about 400 per cent since 1996, and has multiplied by approximately three 
since 2000. A conflation of factors helps explain this evolution. First, mi- 
gration chains and networks — especially within older immigrant commun- 
ities in the city, such as Moroccans or Peruvians — made Madrid a magnet 
for the incoming migration flows at a time when the Spanish economy was 
expanding in low-skilled sectors. Second, a large part of the immigration 
boom was due to the economic and political crisis in Ecuador at the end of 
the 1990s — which was responsible for a substantial share of the total 
growth in immigrant population not only in Madrid but in the whole coun- 
try, facilitated by the lack of requirement of a visa to enter Spain. 

In terms of the composition of the immigrant population, in 1996 im- 
migration flows to the city of Madrid were primarily formed by EU 
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Figure 7.1 Foreign-born population registered in the city of Madrid, 1996-2006 
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citizens. Out of the 241,970 foreigners with residence permits in Spain in 
1985, 65.5 per cent were European, a large majority of them nationals 
from ECC countries. The rest were mainly Moroccans and political refu- 
gees who had fled from dictatorships in Latin America. Although citizens 
from the latter world region still form a substantial share of the total im- 
migrant population living in Madrid, flows from Latin American countries 


Table 7.1 Ten largest immigrant groups in the city of Madrid, 1996 and 2006 


2006 1996 
Country of origin Percentage of Number of Country of origin Percentage of Number of 
total city inhabitants total city inhabitants 
foreign-born foreign-born 
population population 
Ecuador 21.1 124,949 Morocco 13.8 14,794 
Colombia 7.8 45,759 France 8.7 9,347 
Peru 7.1 41,866 Peru 7.4 7,951 
Romania 6.7 39,646 Argentina 7.0 7,475 
Morocco 5.7 32,741 Germany 5.8 6,165 
Bolivia 5.4 31,919 Cuba 4.9 5,007 
Dominican Rep. 4.8 28,394 Dominican Rep. 4.2 4,519 
Argentina 4.1 23,947 Portugal 3.6 3,895 
China 3.8 21,694 Philippines 3.4 3,679 
France 2.3 13,517 Colombia 3.1 3,368 
Total 10 largest 68.8 61.9 
Total 100 589,179 106,772 


Source: Population Register (‘Padrón’) 
Note: Countries in italics are those corresponding to our study groups 
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have substantially changed over time in terms of their internal composi- 
tion (Table 7.1) 

Overall, immigration coming from the four Latin American countries ana- 
lysed in our survey (Ecuador, on the one hand, and Peru, Colombia and 
Bolivia on the other) represented in 2006 almost 43 per cent of the total for- 
eign-born population in Madrid, whereas Moroccan immigration only repre- 
sented 5.7 per cent. Transnational network links have obviously contributed 
to this transformation in the composition by country of origin within the im- 
migrant population of Madrid. While immigrants coming from Latin 
America have traditionally settled in Madrid rather than in other Spanish 
destinations such as Barcelona or Alicante, the opposite happened with re- 
gard to Moroccan immigration, where a larger pre-existing Moroccan com- 
munity resided and was reinforced to a greater extent by migration chains. 


Description of the study and the survey 


As mentioned in previous sections, in the context of the 
LOCALMULTIDEM project a survey of 1,170 individuals was undertaken 
in Madrid. The sample design was stratified by the country of birth of the 
individuals, with the aim of obtaining around 300 individuals for each of 
four different groups: autochthonous Spanish, Ecuadorian, Moroccan, and 
a mixed group of other Andean countries (Bolivia, Colombia and Peru). 
Individuals were selected on the basis of their country of birth, and not 
their nationality, because we wanted to include naturalised immigrants as 
well. Thus, we obtained from the statistical office of the municipality of 
Madrid a simple random sample of individuals who were born in Spain, 
Ecuador, Morocco, Bolivia, Colombia and Peru, respectively. 2 

Fieldwork took place between January 2007 and February 2008, and 
was mostly organised in-house by a fieldwork network purposefully set up 
for this study in Madrid by the University of Murcia (UoM). However, se- 
vere difficulties and delays in fieldwork progression forced us, in some 
cases, to adapt the sampling methods to include various modes of respond- 
ent selection. The gross sample included 2,400 named individuals — 600 
per group. In the end, only 608 interviews were obtained with the origi- 
nally sampled individuals. An additional 115 interviews were obtained by 
substitution within the dwellings of the sampled individuals, 180 inter- 
views for the autochthonous Spaniards were obtained through random 
routes, and 267 interviews were achieved through spatial sampling for the 
Moroccan group — and to a smaller degree the Latin American groups. The 
reasons for employing multiple sampling methods are explained below at 
greater length, but they were mostly related to sampling frame deficiencies, 
the excessive length of fieldwork, the higher rates of ineligible sample 
units extracted, and the lack of time to get additional nominal sample units 
from the local statistical office. 
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It is important to highlight, though, that despite the use of different sam- 
pling methods the results we obtained in terms of the socio-demographic 
profile in our sample are strikingly similar to the known distribution of 
some of these variables within the population register and to the figures 
one can obtain for these same groups from the 2007 Spanish National 
Immigrant Survey (analyses not shown here). Hence, the employment of 
various sampling methods to respond to fieldwork difficulties has not 
biased our sample in socio-demographic terms. Moreover, several substan- 
tive analyses we have carried out elsewhere (Morales, Anduiza, Rodríguez 
& San Martín 2008, 2010) found no significant effect of the sampling 
method on a number of different variables related to the socio-political atti- 
tudes and behaviours of the respondents. 

Table 7.2 describes the sample structure of the survey and Table 7.3 pro- 
vides the descriptives of the main socio-demographic characteristics of the 
interviewed respondents. All interviews were conducted face-to-face and 
completion of the questionnaire took on average approximately 50 minutes. 
The respondents of the Moroccan group could choose to do the interview 
in Spanish or in Arabic, and a number of bilingual interviewers were avail- 
able for this.'* This aspect was crucial, as many Moroccan respondents 
were not able to communicate in Spanish with sufficient fluency. 

As mentioned in the first pages of this chapter, one important aspect of 
the LOCALMULTIDEM survey conducted in Madrid is that — as far as it 
was feasible — it tried to replicate the same fieldwork procedures and strat- 
egies put in place for the ESS in Spain.'* To this end, a contact form that 
replicated — with minimal necessary improvements — those employed in 
the Spanish fieldwork operations of the ESS was used for the 
LOCALMULTIDEM survey in Madrid. Interviewers were trained by the 
UoM team with a handbook that contained very similar materials to those 


Table 7.2 Description of the gross and net samples 


Original Interviews Interviews Interviews Interviews Total 
nominal nominal replacement from spatial from random interviews 
sample sample from original sampling routes 
sample 
Spanish 600 91 19 0 180 290 
‘False’ Moroccan 107 16 1 0 17 
True Moroccan 493 95 32 171 298 
Ecuadorian 600 232 32 27 291 
Mixed Andean 600 177 31 69 277 
Bolivia 162 52 12 15 79 
Colombia 219 58 9 35 102 
Peru 219 67 10 19 96 
Total 2,400 608 115 267 180 1,170 


Sources: LOCALMULTIDEM and CAPSOCINMIG survey, 2007 


158 LAURA MORALES AND VIRGINIA ROS 


employed by the ESS team in Spain. A letter with the UoM logo present- 
ing the survey and announcing the visit of the interviewers was sent one 
month in advance of the starting date of fieldwork.'® As in the ESS in 
Spain, interviewers were offered a number of salary incentives that were 
linked to their results, so that the baseline payment was increased gradually 
with higher response rates achieved per assignment.'’ Unlike the ESS we 
had no budget for monetary incentives for respondents, so our results need 
to be interpreted in light of this (possibly) important difference. Before an 
individual in the sample was abandoned as ‘unproductive’ by the inter- 
viewer, four visits, including one in the evening (after 8pm) and another 
during the weekend, were required; and the fieldwork coordinator would 
reassign the case to another interviewer so that at least another visit in the 
evening and on a weekend would take place. Fortnightly reports of field- 
work progress were produced by the fieldwork coordinator. 

As will become clear in the next sections, a number of fieldwork diffi- 
culties were encountered that required some adaptations in strategies and 
sampling methods. First, response rates for the autochthonous Spanish pop- 
ulation were extremely low and distracted interviewers from concentrating 
on the difficult task of locating immigrant respondents in their house- 
holds.'* This led us to change the initial plans and complement the final 
sample with random route interviews conducted by a professional survey 
institute for 180 autochthonous individuals. 

Second, once fieldwork was under way, we discovered that almost 20 
per cent of the original sample for the Moroccan group was in fact what 
we termed ‘false’ Moroccans: autochthonous Spanish who were born in 
territory that is now Morocco but was a Spanish colonial enclave when 
they were born.” This meant that, all of a sudden, a large part of our sam- 
ple could not be used, and response rates for Moroccans turned out to be 
insufficient to reach or approximate the 300 individuals required. Hence, 
we decided to substantially complement the sample with spatial sampling 
that was conducted in a number of locations — immigration information of- 
fices, public spaces in the neighbourhoods and, above all, the queue out- 
side the Moroccan consulate. In a few cases, we did this as well for the 
Latin American groups to complement our sample, particularly as a result 
of the high refusal rates of the Colombians. 

Third, for all immigrant groups — but especially for the Latin American 
ones — it was quite common that the sampled individual no longer lived in 
the dwelling but that individuals of the same national origin did. Whenever 
the sampled individual had moved within the city, interviewers were in- 
structed to try to get the new address and attempt a new contact. However, 
this was not feasible in many cases, and hence during fieldwork we de- 
cided to allow for substitution within the dwelling or the adjacent dwell- 
ings wn individuals of the same country of origin, gender and age 
group. 
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Table 7.3 Main descriptives of the net sample in Madrid 


ES MA EC AND 
(n=307) — (n=298)  (n=291)  (n=277) 
Percentage female 51 39 55 59 
Age 
16-25 12 18 18 17 
26-35 15 49 40 33 
36-45 16 25 25 27 
46-55 17 6 12 16 
56-65 14 2 4 4 
65+ 26 0 1 3 
Education 
Illiterate or primary not completed 17 18 4 3 
Primary completed 37 50 55 36 
Secondary completed 22 24 30 40 
University degrees 23 8 11 21 
Religious attendance 
Atheist/never attends 44 38 22 31 
Attends at least once a year 31 57 60 44 
Attends at least once a week 25 5 18 25 
Percentage Muslim - 98 - - 
Years since arrival in Spain 
Two years or less - 14 2 12 
3-5 years - 22 11 32 
6-10 years - 43 76 41 
More than 10 years - 21 11 15 
Mean value - 8 8 7 
Administrative situation 
Short-term permit 5 years or less - 70 53 44 
Long-term permit more than 5 years - 10 13 8 
Renovating expired permit - 8 23 17 
Never had a permit - 11 3 23 
Percentage Spanish citizenship 100 11 18 20 


Sources: LOCALMULTIDEM and CAPSOCINMIG survey, 2007 
Legend: ES=Spanish, MA=Moroccan, EC= Ecuadorian, AND=Andean 


Finally, the fact that we decided to conduct the fieldwork mostly in- 
house and with a purposefully built fieldwork network resulted in addition- 
al difficulties related to high interviewer turnover and limited experience 
with nominal samples — which are not that common in Spain. 

Although, of course, we do not claim to have been able to replicate the 
efforts and success of the fieldwork operations of the ESS in Spain — given 
the substantial difference in funding — we want to highlight that our field- 
work efforts and reach were not very far from those achieved in the first 
round of the ESS in Spain completed in 2002, as shown when comparing 
the number of visits received by the units (Table 7.4). 

In the next sections we describe in detail how successful these efforts 
were in terms of achieving high responses rates and how they compare 
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Table 7.4 Number of visits the units received, percentage over total sample 


assigned 
ESS Spain LOCALMULTIDEM Survey, 2007-2008 

2002 2006 ES MA EC BO CO PE 
0 1.5 0.0 3.3 1.9 1.8 1.7 2.8 2.5 
1 36.9 18.7 42.4 60.9 32.3 37.5 34.5 28.0 
2 20.6 24.5 16.8 14.4 15.7 18.2 17.1 19.9 
3 13.2 18.0 10.6 11.6 15.2 13.6 13.5 13.6 
4 9.4 13.9 9.7 4.3 12.0 11.9 10.7 11.0 
5 9.5 9.8 6.6 1.7 7.2 6.3 7.1 8.1 
6 4.2 6.6 5.1 1.9 6.4 3.4 5.6 3.4 
7 2.9 3.7 3.1 1.3 4.6 4.0 5.2 5:5 
8 1.2 1.7 1.4 1.3 3.4 1.7 2.0 3.4 
9 0.6 1.2 0.5 0.4 1.0 0.6 0.8 1.7 
10 or more 0.2 2.2 0.6 0.2 0.6 1.1 0.8 3.0 
Total 100 100 100 100 100 100 100 100 
N 3,702 3,290 885 466 626 176 252 236 
Minimum 0 1 0 0 0 0 0 0 
Maximum 10 17 14 10 15 12 13 15 
Average 2.64 3.38 3.00 2.38 3.16 2.89 3.23 3.50 


St. deviation 1.89 2.21 2.14 1.83 2.26 2.14 2.26 2.69 


Sources: Authors' own elaboration with data from Riba, Torcal and Morales (2010) for the 
2006 Spanish ESS and from the LOCALMULTIDEM survey in Madrid 

Legend: ES=Spanish, MA=Moroccan, EC=Ecuadorian, BO=Bolivian, CO=Colombian, 
PE=Peruvian 


with the high quality benchmark of the ESS in Spain. In this sense, we will 
mostly focus on the sample that pertains to the nominal random sample 
originally issued, as this is what is comparable with the ESS figures; but 
we will occasionally give some background information about the success 
and effort involved in the other sampling strategies. Finally, we will from 
now on be using the 2006 ESS Spanish survey as our comparative stand- 
point, as this was the one fielded closest to the time when our survey was 
also conducted, even if this means that we are drawing a comparison with 
the third round of the survey in Spain, rather than the first — which would 
be a fairer comparison to our own survey. 


7-4 Comparing the response rates of migrant-background and 
autochthonous individuals 


In this section we focus on providing an overall view of the outcome pat- 
terns for all contact attempts with individuals and households. First we 
concentrate on the original nominal sample and on those sample units for 
which at least one visit was attempted.2! Table 7.5 compares the overall re- 
sults of the LOCALMULTIDEM in Madrid with the 2006 ESS in Spain.” 
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Table 7.5 Fieldwork results in Madrid (nominal sample only) compared to the ESS 


ESS Spain 2006 LOCALMULTIDEM  LOCALMULTIDEM (groups, %) 


(overall) 

N % N % ES MA EC BO CO PE 
Completed 1,876 57 726 36 22 44 45 40 32 36 
interviews 
Refusal 559 17 350 17 33 14 10 6 12 13 
No contact 225 7 276 14 19 10 10 1 14 15 
Unavailable 167 5 193 9 13 9 7 9 8 9 
Ineligible 430 13 423 21 11 20 25 3 3l 23 
Appointment 0 0 6 0.3 0.2 1 02 0.6 0 0 
not followed 
Other, invalid 33 1 53 3 2 2 4 2 2 3 
or incomplete 
Total 3,290 100 2,027 100 100 100 100 100 100 100 


(578) (286) (586) (158) (208) (211) 


Sources: Authors’ own elaboration with data from Riba, Torcal and Morales (2010) for the 
2006 Spanish ESS and from the LOCALMULTIDEM survey in Madrid 

Legend: ES=Spanish, MA=Moroccan, EC=Ecuadorian, BO=Bolivian, CO=Colombian, 
PE=Peruvian 


First, we notice that the LOCALMULTIDEM survey had a substantially 
smaller proportion of completed interviews than the ESS. However, this is 
not due to higher refusal rates, but to substantially higher non-contact rates 
and ineligible cases, which are always higher in urban areas due to geo- 
graphical mobility and lifestyle patterns. Second, it is important to under- 
score that refusal rates varied substantially across origin groups, with more 
than double the amount of refusals among Spanish than all immigrant 
groups.” Third, surprisingly given the contrasting evidence reviewed in 
the literature, non-contact rates also tended to be higher among Spanish 
and among the two migrant groups with a longer history of migration to 
Spain: Colombians and Peruvians. Finally, it is the much higher rates of in- 
eligible cases among migrants — especially the Latin American ones — that 
explain the lower success rates in our survey. There were between two and 
three times more ineligible cases among the migrant groups than among 
the Spanish, and our percentage of ineligible cases for the Spanish is very 
similar to that produced by the ESS, hence rendering greater credibility to 
our results among immigrants. 

The stark contrast between the autochthonous Spanish and immigrant 
groups is even more evident when we compare the final response rates as 
calculated with the ESS standard formulae.”* As we see in Table 7.6, the 
‘true’ and ‘fieldwork’ response rates that exclude ineligibles from the cal- 
culations provide encouraging results for several immigrant groups, in par- 
ticular Ecuadorians and Bolivians — two of the larger Latin American 
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Table 7.6 Final response rates, nominal sample only 


ESS Spain 
2006 LOCALMULTIDEM 
ES MA EC BO CO PE 
Gross response rate 57.0 22.0 44.4 45.1 40.5 32.2 36.5 
“True' response rate 65.6 24.6 55.2 59.7 58.2 46.9 47.5 
‘Fieldwork’ response rate 68.1 27.0 57.7 61.4 61.0 479 49.7 


Sources: Authors” own elaboration with data from Riba, Torcal and Morales (2010) for the 
2006 Spanish ESS and from the LOCALMULTIDEM survey in Madrid 

Legend: ES=Spanish, MA=Moroccan, EC=Ecuadorian, BO=Bolivian, CO=Colombian, 
PE=Peruvian 


groups in the city of Madrid. We should note, additionally, that the re- 
sponse rate for the ESS in the region of Madrid was lower (53 per cent), 
and even more for Madrid city. 

Although we resorted to complementary sampling methods, Table 7.7 
shows that these did not necessarily require less effort on the part of inter- 
viewers, but they were often quicker to handle within the time constraints 
of the need to complete the study. In particular, substitution within the 
dwelling or the adjacent dwellings when ineligibility was due to the indi- 
vidual having moved elsewhere (or not having lived there at all) also re- 
sulted in high failure rates. Random routes for the autochthonous popula- 
tion also suffered from very high rates of failed contacts, especially due to 
non-contacts and ineligibility. Spatial sampling proved to be relatively suc- 
cessful, as refusal rates were low — especially among the individuals who 
were queuing for long hours outside the consulate premises — but often the 
incapacity to locate anyone in certain public spaces required changing the 
intersection points. 

Thus, overall, with sufficient time for conducting fieldwork and a strong 
fieldwork network of professional interviewers, the success rates of nomi- 
nal sampling seem to outperform other alternatives. Only in the absence of 
sufficient time or resources would we recommend spatial sampling of im- 
migrants in Spain. 


7.5 Does trying harder pay off? The success of refusal 
conversion and additional location attempts 


The general recommendation in the survey research literature and, in par- 
ticular, the strategy followed by the ESS is to try achieving response rates 
as close as possible to 70 per cent by increasing efforts to locate respond- 
ents and by attempting to convert refusals, especially “soft” ones. In this 
section we focus on the results of following that advice for the 
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LOCALMULTIDEM survey in Madrid and on how its results compare 
with those obtained by the ESS team in Spain. 

In Table 7.8 we see that the average and median number of visits was in 
most cases very similar to that put in place by the ESS fieldwork institute 
in Spain. Equally, except for the case of Moroccan individuals, the number 
of individuals that received five or more visits fluctuated around 25 per 
cent, as in the ESS. However, given that our survey was conducted in the 
largest city in Spain — as opposed to the nationwide figures available for 
the ESS — successful contact is much lower for all cases and, particularly, 
for the Spanish sub-sample. In addition, refusals in at least one of the visits 
are much lower for all immigrant groups than for the Spanish. 

We attempted refusal conversion whenever there was some chance of 
success. This was more often the case among the immigrant groups than 
among the Spanish, an indication of how uncooperative the autochthonous 
population has become in urban areas in Spain. Soft refusals were more 
common among the Ecuadorians, and hence they were the target of more 
conversion attempts. Given these different rates of conversion attempts, it 
is not surprising that there were considerable differences in the extent to 
which they were successful. Between 40 and 100 per cent of the conver- 
sion attempts ended in a completed interview. Success was highest among 
the Bolivians, but they were the least exposed to such attempts; success 
was lowest among Ecuadorians and Colombians, but the former were sub- 
jected to conversion attempts in a greater proportion. The main corollary is 
that these attempts at converting refusals into interviews actually work, but 
not necessarily more among the immigrant population than among the 
Spanish. 

Similarly, we examined the success of additional contact attempts be- 
yond the standard four visits recommended by the ESS international coor- 
dination team (Table 7.9). As can be seen, the Spanish were more difficult 


Table 7.9 Efforts and success in locating respondents, nominal sample only 


ESS Spain LOCALMULTIDEM 
2006 


ES MA EC BO CO PE 


Units resulting in no contact with 7 14 8 8 7 11 1⁄1 
anyone after four visits (%) 

Non-contacts after four visits in units 87 82 92 89 73 100 92 
that received more than four visits (%) 

Successful interviews after four visits 34 6 27 25 25 22 4 


(if no previous contact) (%) 


Sources: Authors’ own elaboration with data from Riba, Torcal and Morales (2010) for the 
2006 Spanish ESS and from the LOCALMULTIDEM survey in Madrid 

Legend: ES=Spanish, MA=Moroccan, EC=Ecuadorian, BO=Bolivian, CO=Colombian, 
PE=Peruvian 
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Table 7.10 Comparing the results of fieldwork efforts, nominal sample only (%)* 


ESS Spain LOCALMULTIDEM 
2006 


ES MA EC BO CO PE 


Interview obtained with normal 62 26 60 65 67 49 53 
protocols 

Refusal converted into interview 6 3 2 2 1 2 2 
Non-contact converted into 2 1 3 3 2 4 1 
interview after fourth visit 

Final refusal 21 45 20 15 11 21 20 
Final visit no contact 9 25 14 16 19 24 23 
Total 100 100 100 100 100 100 100 


Sources: Authors” own elaboration with data from Riba, Torcal and Morales (2010) for the 
2006 Spanish ESS and from the LOCALMULTIDEM survey in Madrid 

* The figures of LOCALMULTIDEM are calculated excluding units that were ineligible, 
unavailable or pending an appointment in the last visit 

Legend: ES=Spanish, MA=Moroccan, EC=Ecuadorian, BO=Bolivian, CO=Colombian, 
PE=Peruvian 


to contact after the four standard visits, while a number of immigrant 
groups show patterns that are more similar to the nationwide averages pro- 
vided by the ESS in Spain. In the vast majority of cases where no contact 
was achieved, extra visits were programmed, following standards applied 
by the Spanish ESS team. However, extra visits were unevenly successful 
across groups and systematically less so than they were for the ESS in 
Spain in 2006. The Spanish and the Peruvians are particularly difficult 
groups to locate, whereas the remaining four immigrant groups show simi- 
lar success rates of around 25 per cent. 

All things considered, refusal conversion and extra contact attempts are 
only moderately successful and result in a meagre four percentage points 
of additional completed interviews for the Spanish and between three and 
six extra percentage points for the immigrant groups studied (the sum of 
rows 2 and 3 in Table 7.10). Hence, its use needs to be considered care- 
fully and in relation to how much pressure it will put on the fieldwork 
organisation. 


7-6 Concluding discussion 


The purpose of this chapter was to evaluate how response rates to a social 
survey might differ between autochthonous and immigrant populations in 
an urban setting in Spain. We presented information from the contact forms 
collected during fieldwork for the LOCALMULTIDEM survey in Madrid. 
This was compared with the equivalent results produced by the ESS in 
Spain. LOCALMULTIDEM is the first such large-scale survey conducted 
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of an extensive sample of the immigrant population in a single major city 
in Spain, and one of the first large-scale socio-political surveys conducted 
of immigrants in Spain attempting to use nominal random sampling. As 
such, this should be viewed as a first promising step forward in what 
should become now increasingly common in Spain. Hence our findings re- 
garding fieldwork operations should be of use to other social scientists de- 
signing similar surveys in similar contexts. 

A number of conclusions can be derived from our findings. First, even 
if funding cannot match the resources deployed by the ESS, it is possible 
to conduct a survey employing nominal random sampling from population 
registers with immigrant populations in Spain. Our study shows that it is 
feasible, and it suggests that using a professional and well-established 
fieldwork organisation should substantially improve results. In most cases, 
our failures were related more to the instability of our network of inter- 
viewers than to cost issues — though funding is certainly important and we 
would have obtained better results had we been able to pay interviewers 
more. 

Second, the main obstacle to obtaining high response rates from immi- 
grant populations is not related primarily to non-contacts or refusals, but to 
the relatively high proportion of ineligible cases one will encounter. In this 
regard, our results are in line with those that indicate that immigrants are 
usually more cooperative than the autochthonous population, but our find- 
ings are at odds with those that indicate that immigrants are more difficult 
to locate. In our case, the fact that we were controlling for ‘urbanicity’ — as 
both autochthonous and immigrant individuals were located in the same 
highly urbanised city — and that immigrant households are, on average, in- 
habited by more people than those formed by autochthonous Spanish, 
counteracted the longer average hours of work of the immigrant popula- 
tion. Thus, any survey that aims at studying the immigrant population in 
Spain using random samples of population register individual records 
should take this into consideration and obtain samples that are approxi- 
mately two and a half times larger than the final sample they want to 
achieve. In addition, if comparisons with an autochthonous population 
group are to be included, sample preparations should take into account the 
very high rates of refusals and non-contacts that the Spanish display in ur- 
ban settings, as this implies issuing samples at least four times larger than 
the desired number of completed interviews. 

Third, because situations of ineligibility among the immigrant popula- 
tion — especially due to relocation or to fictitious registration in the popula- 
tion register — are relatively common, it is advisable to contemplate either 
substitutions within the dwelling and adjacent dwellings, or to design the 
samples as household samples rather than individual samples — as was 
done in the Spanish immigrant survey carried out in 2007 (ENI, for more 
details see chapter 3 in this volume). Of course, this is an alternative to 
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simply extracting a larger sample, but one that might facilitate fieldwork 
operations in certain cases. 

Finally, our results have provided evidence of the existence of different 
patterns of response rates across immigrant groups, but in most cases the 
largest gap is with the autochthonous population, clearly indicating that 
surveys that aim to combine responses from both will have to design field- 
work strategies tailored to the considerably different survey response be- 
haviours of these groups. This notwithstanding, we believe that there are 
grounds for optimism and that good fieldwork planning should help over- 
come most of the obstacles and result in achieving relatively high response 
rates from immigrants in Spain, at least for the next few years. 


Notes 


1 The data for this chapter have been collected, processed and analysed with the support of 
the LOCALMULTIDEM project (CIT5-CT-2005-028802) funded by the European 
Commission's 6th Framework Programme, and the CAPSOCINMIG Project (SEJ2005- 
07733/CPOL) funded by the Spanish Ministry of Science and Education. Further informa- 
tion on the projects can be found on their websites: www.um.es/LOCALMULTIDEM and 
www.um.es/capsocinmig. A previous version of this chapter was presented at the ESF 
Exploratory Workshop ‘Surveying Immigrant Population in Studies of Social and 
Political Participation: Methodological and Technical Challenges” organised in Madrid. 
We thank the participants in that workshop, the volume editors and the three IMISCOE 
anonymous reviewers for their thoughtful comments on previous versions of this chapter. 
The authors also wish to sincerely thank Elisa Rodríguez — the fieldwork coordinator in 
Madrid — and all interviewers for their hard work for this study. Laura Morales closely 
monitored fieldwork preparations and progress throughout the study, and Virginia Ros 
was part of the fieldwork and interviewing operations, so they know very well how diffi- 
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2 See Cebolla and González-Ferrer (2008) for a thorough introduction to recent immigration 
trends and policies in Spain. 

3 Until the Spanish National Immigrant Survey was conducted in 2007, there was no com- 
prehensive survey information on the immigrant population in Spain (see chapter 3 in this 
volume). 

The samples are described in greater detail in Morales and Giugni (2011). 

5 However, see Groves (2006), Keeter et al. (2000) and Keeter et al. (2006) for illustrations 
of how substantially different response rates due to varying fieldwork strategies need not 
bias the distribution of responses on a number of attitudinal and behavioural indictors. 
Groves and Peytcheva (2008) conduct a meta-analysis of the factors that drive response 
bias in relation to high non-response. 

6 The effective sample size is the size of a simple random sample which would produce the 
same precision (standard errors) as the design actually used. 

7 These figures are available from the ESS documentation reports for each of the rounds 
that can be found on <http://ess.nsd.uib.no/ess/>. 

8 This is the case for Belgium, Denmark, Germany and Hungary, where the sample frames 
do not include this population. The documentation is insufficiently clear in this regard for 
the Czech Republic and Sweden. 
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One limitation, though, of Blohm and Diehl’s study of immigrants in Mannheim is that 
they did not include a sub-sample of the autochthonous German population in their sur- 
vey and, hence, their comparisons are with the German ALLBUS nationwide results. 
Higher response rates in the Mannheim study could, possibly, have affected autochtho- 
nous Germans as well, so there is a certain amount of uncertainty in their conclusions 
about overall response rates from immigrants. 

Nevertheless, the results by Feskens et al. (2007) indicate that a large part of the lower 
contact and response rates of immigrant minorities are due to their living disproportion- 
ately in highly urbanised areas. 

See also Sasao (1994). 

The current system of population registers started in 1996. Statistical information on the 
immigrant population in Spain before this date is poor and unreliable. Therefore, we only 
report on changes in the size and composition of the immigrant population in Madrid be- 
tween May 1996 and July 2006, the relevant date prior to our survey. 

Once the individuals were located for interview, the country of birth of their parents was 
checked. For the autochthonous group, any individual born in Spain with both parents 
born elsewhere than the other five foreign countries included in the study qualified as an 
“autochthonous” individual. If any of the parents had been born in any of the other five 
countries, then this individual was deemed as a second-generation migrant and relocated 
to the corresponding group by the national origin of the parent. Equally, when we found 
individuals who had been born in these five countries but of both parents born in Spain, 
they were relocated to the “autochthonous” group. The latter situation was very frequent 
for the sample of individuals born in Morocco, as around 20 per cent of our Moroccan 
sample turned out to be autochthonous population that had been born in the former colo- 
nial cities of North Africa under Spanish rule. The unanticipated magnitude of this prob- 
lem resulted in serious problems with the Moroccan sample for which we had to adapt 
our sampling methods. 

Spanish is the main language of the studied Latin American groups, and we did not en- 
counter any case of respondents of indigenous origin who were not fluent in Spanish be- 
cause of Quechua or Aymara ancestry. 

This was facilitated by the fact that Laura Morales — who coordinated the 
LOCALMULTIDEM project — had been a member of the Spanish ESS coordination team 
between 2001 and 2007 and was well acquainted with fieldwork organising and monitor- 
ing practices for the ESS in Spain. 

In this aspect our operations were more modest than those in place for the ESS, as the 
larger funding of the latter allows for three different deliveries: a first one two months pri- 
or to the commencement of fieldwork, another two weeks prior to the starting dates of in- 
terviews that also includes a leaflet about the ESS in Europe, and a third one targeted spe- 
cifically for the non-contacts and the refusals. 

The following ranges were used for the incentives: up to 30 per cent, between 30 and 39 
per cent, between 40 and 49 per cent, between 50 and 55 per cent, and over 56 per cent. 
Furthermore, interviewers were presented with two payment models, one with lower base- 
line and maximum payments but with a per unit payment just for completing the contact 
forms, and another with higher baseline and maximum payments but with no per unit 
payment. They preferred the latter. 

Response rates in the region of Madrid were consistently and repeatedly the lowest (to- 
gether with those in Catalonia) for the ESS in Spain. For the 2006-2007 survey of the 
third round of the ESS, the gross response rate for the region of Madrid was 53 per cent, 
but in previous rounds it had been closer to 45 per cent. Yet, this is for the whole region, 
while our survey was conducted only in the capital city of Madrid. Riba et al. (2010: 
Tables 13, 14 and 15) show that the response rates for large cities are often half that of 
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the overall sample. Hence, our very low response rates for the autochthonous population 
are not very surprising, and are similar to those reported in that work for large cities. 

In fact, we believe that in many cases this is due to malpractice by the local population 
registers, as back then these areas were part of Spain and, hence, the country of birth 
should have been registered as Spain and not Morocco. 

Interviewers were given the following instructions: (1) when they encountered an ineli- 
gible unit (the individual had moved to another city, the individual was unknown to the 
current dwellers, the individual had moved to an unknown destination, the individual had 
moved to an institutional residence), interviewers were allowed to attempt a substitution 
within the dwelling they were visiting if the current dwellers were of the same national 
origin as the sampled individuals, and were instructed to select the potential respondent 
with the last birthday rule; (2) when they encountered a refusal or a non-contact, inter- 
viewers were allowed to seek a substitution with respondents of the same national origin 
in the first instance among the dwellings of the same building and, failing that, the two 
adjacent buildings in the street — for example if the sampled dwelling was in number 17, 
they could seek substitutions in numbers 15 and 19 — and were to use the last birthday 
method to select respondents in dwellings with multiple occupants. 

The sample units for which no attempt to visit was made are, primarily, of individuals 
coming from the Spanish and Moroccan sub-samples. In the case of Moroccans, this was 
due to the high number of “false” Moroccans — as described above — that were in reality 
not eligible under that sub-sample. In the case of the Spanish, the high non-response rates 
forced us to commission an external fieldwork institute to conduct random routes at a 
quicker pace, which meant that some — though few — of our original sampled units were 
never approached because the in-house fieldwork efforts were diverted to completing the 
immigrant sub-samples. 

It is important to note that we are comparing our results in the city of Madrid with the 
overall sample results for the whole country in the ESS. The reader must be warned that 
response rates are substantially lower for urban areas in Spain and that, in particular, 
Barcelona and Madrid are especially complicated areas for fieldwork with much lower re- 
sponse rates than obtained nationwide (see Riba et al. 2010). 

Again, the high refusal rates for Spanish individuals are in line with the results for large 
metropolitan cities found in Riba et al. (2010). 

The gross response rate is the total number of valid interviews over the total sample that 
was actively pursued during fieldwork. The ‘true’ and ‘fieldwork’ response rates are cal- 
culated as per the ESS technical definitions: the ‘true’ response rate is the number of com- 
pleted and valid interviews divided by the number of units sampled minus pure ineligible 
units (deceased respondents, respondents moved out of the country/city studied, derelict 
or demolished addresses, addresses corresponding to buildings not yet built or not ready 
for occupation, non-residential addresses, addresses not occupied, individuals unknown in 
the dwelling or who have never really resided there). The ‘fieldwork’ response rate also 
subtracts units that are ineligible in practical terms (respondents who are away during the 
whole fieldwork period, respondents mentally or physically unable to participate, respond- 
ents with language problems, or dwellings for which the address was not traceable or in- 
sufficient). In the LOCALMULTIDEM sample there were no sampled individuals with 
language barriers, as this would have only applied to the Moroccan group and the ques- 
tionnaire was translated into Arabic for this very reason. 

As mentioned before, in papers analysing the substantive responses provided by respond- 
ents we have been able to confirm that the sampling method has no significant effect on 
our variables of interest (Morales et al. 2008). 


COMPARING AUTOCHTHONOUS AND MIGRANT RESPONSE 171 


References 


Billiet, J., M. Philippens, R. Fitzgerald & I. Stoop (2007), ‘Estimation of nonresponse bias in 
the European Social Survey: using information from reluctant respondents’, Journal of 
Official Statistics 23 (2): 135-162. 

Blohm, M. & C. Diehl (2001), ‘Wenn Migranten Migranten befragen. Zum Teilnahme- 
verhalten von Einwanderern bei Bevólkerungsbefragungen”, Zeitschrift für Soziologie 30 
(3): 223-242. 

Bradburn, N.M. (1985), “Response effects”, in P.H. Rossi, J.D. Wright & A.B. Anderson 
(eds), Handbook of survey research. San Diego: Academic Press. 

Bronner, A.E. (1988), ‘Surveying ethnic minorities’, in W.E. Saris & LN. Gallhofer (eds), 
Sociometric Research. London: MacMillan. 

Cebolla, H. & A. González-Ferrer (2008), La inmigración en España (2000-2007). De la 
gestión de los flujos a la integración de los inmigrantes. Madrid: Centro de Estudios 
Políticos y Constitucionales. 

Deding, M., T. Fridberg & V. Jakobsen (2008), ‘Nonresponse in a survey among immigrants 
in Denmark”, Survey Research Methods 2 (3): 107-121. 

Feskens, R., J. Hox, G. Lensvelt-Mulders & H. Schmeets (2007), ‘Nonresponse among ethnic 
minorities: A multivariate analysis”, Journal of Official Statistics 23 (3): 387-408. 

Feskens, R., J. Hox, G. Lensvelt-Mulders & H. Schmeets (2006), ‘Collecting data among eth- 
nic minorities in an international perspective”, Fields Methods 18: 284-304. 

Feskens, R., J. Hox, H. Schmeets & W. Wetzels (2008), “Incentives and ethnic minorities: 
Results of a controlled randomized experiment in the Netherlands”, Survey Research 
Methods 2 (3):159-165. 

Fowler, F.J. (2002), Survey research methods. 3rd ed. Thousand Oaks: Sage. 

Groves, R.M. (2006), “Nonresponse rates and nonresponse bias in household surveys”, Public 
Opinion Quarterly 70 (5): 646-675. 

Groves, R.M. 8 E. Peytcheva (2008), ‘The impact of nonresponse rates on nonresponse bias: 
A meta-analysis’, Public Opinion Quarterly 72: 167-189. 

Keeter, S., K. Courtney, M. Dimock, J. Best & P. Craighill (2006), “Gauging the impact of 
growing nonresponse on estimates from a national RDD telephone survey”, Public 
Opinion Quarterly 70: 759-779. 

Keeter, S., C. Miller, A. Kohut, R.M. Groves & S. Presser (2000), ‘Consequences of reducing 
nonresponse in a national telephone survey’, Public Opinion Quarterly 64: 125-148. 

Morales, L., E. Anduiza, E. Rodriguez & J. San Martin (2010), ‘La participacion politica de 
los inmigrantes en Barcelona y Madrid: Comportamiento electoral y accion politica’, in 
D. Moya & A. Viñas (eds), Sufragio y participación politica de los extranjeros extraco- 
munitarios en Europa. Barcelona: Fundació Pi i Sunyer. 

Morales, L., E. Anduiza, E. Rodriguez & J. San Martin (2008), ‘Capital social, pautas identita- 
rias y actitudes hacia “los otros”: La incorporación civica de la población de origen inmi- 
grante en Barcelona y Madrid’, Panorama Social 8: 119-142. 

Morales, L. & M. Giugni (2011), ‘Political opportunities, social capital and the political inclu- 
sion of immigrants in European cities’, in L. Morales & M. Giugni (eds), Social Capital, 
Political Participation and Migration in Europe. Making Multicultural Democracy Work? 
Basingstoke: Palgrave Macmillan. 

Peytcheva, E. (2008a), ‘Language of administration as a cause of measurement error’, paper 
presented at AAPOR Conference (New Orleans). 

Peytcheva, E. (2008b), Language of administration as a source of measurement error: 
Implications for surveys of immigrants and cross-cultural survey research. Dissertation. 
University of Michigan, Ann Arbour. 


172 LAURA MORALES AND VIRGINIA ROS 


Riba, C., M. Torcal & L. Morales (2010), “Estrategias para aumentar la tasa de respuesta y sus 
resultados en la Encuesta Social Europea en España”, Revista Internacional de Sociología 
68 (3). 

Sasao, T. (1994), “Using surname-based telephone survey methodology in Asian-American 
communities: Practical issues and caveats’, Journal of Community Psychology 22: 283- 
295. 

Stoop, L (2005), The hunt for the last respondent: Nonresponse in sample surveys. The 
Hague: Social and Cultural Planning Office of the Netherlands. 

Stoop, L, J. Billiet, A. Koch & R. Fitzgerald (2010), Improving survey response. Lessons 
learned from the European Social Survey. Chichester: John Wiley & Sons. 

Thomas, M. (2008), “Improving migrant participation in the Labour Force Survey: Non-re- 
sponse and attitudes of non-English speaking migrants to participation”, Survey 
Methodology Bulletin 63: 39-51. 

Van den Brakel, J.A., R. Vis-Visschers & J.J.G. Schmeets (2006), ‘An experiment with data- 
collection modes and incentives in the Dutch Family and Fertility Survey for young 
Moroccans and Turks”, Field Methods 18: 321-334. 


8 Non-response among immigrants in Denmark 


Mette Deding, Torben Fridberg and Vibeke Jakobsen 


8.1 Introduction 


In Denmark, as in other European countries, it has turned out to be very 
difficult to achieve a satisfactorily high participation of different immigrant 
groups in surveys. Both in general population surveys and in surveys spe- 
cifically targeted to immigrant populations, non-response rates are typically 
relatively high, and the basic lesson from these surveys is that interviewing 
immigrants requires considerations other than those applied to interviewing 
the majority population. Nonetheless, few studies have focused on non-re- 
sponse among immigrants (see, e.g. Feskens, Hox, Lensvelt-Mulders & 
Schmeets 2007; Dale & Haraldsen 2000; Van den Brakel, Vis-Visschers & 
Schmeets 2006). 

The main reason for concern about high non-response rates is that non- 
response may generate bias problems. In particular, non-response poses a 
problem if it is correlated with the variables of interest. Previous research 
has shown that while non-response bias occurs, the non-response rate of a 
survey alone is not a good predictor of the magnitude of the bias (Groves 
2006). Blind efforts to reduce non-response may increase the bias prob- 
lems. Instead, efforts at reducing non-response should be guided by knowl- 
edge about the character of the non-response bias and about the ways in 
which groups are affected by efforts to reduce it (Groves 2006). 

A survey on education and labour market affiliation among persons in 
the 18-45 age group from three different immigrant groups (with origins in 
Iran, Turkey and Pakistan) and native Danes, carried out in 2006, provided 
the opportunity to look further into the characteristics of non-respondents 
in the different groups of the population.' Although great effort was made 
in the survey design and data-collection process, the survey more than ful- 
filled the expectation that non-response among the immigrant groups 
would be high — on average the response rate among the three immigrant 
groups was about 20 percentage points lower than for the Danes. 

However, as the sample for the survey was drawn from the Danish pop- 
ulation register, we were able to compare the groups of respondents and 
non-respondents within each of the four population groups on the basis of 
administrative register data available at Statistics Denmark. Furthermore, 
we had access to some information on the interviewers from the survey 
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organisation and to rather detailed data on the number of contacts and in- 
formation on reasons for non-response. By using this information we were 
able to look into the influence of the characteristics of the sample persons 
and the influence of the Interviewers on the non-response. 

The survey collection revealed large differences between the immigrant 
groups in reasons for non-response — for instance, contacting immigrants 
from Pakistan was very difficult, while refusals were a great problem 
among immigrants from both Turkey and Pakistan. For this reason we dis- 
tinguished between non-contacts, refusals and other reasons for non-re- 
sponse, and did separate analyses on how contacts, as opposed to non-con- 
tacts, and cooperation, as opposed to non-cooperation, depend on various 
characteristics of the sample persons and of the interviewers. Different 
types of non-response are likely to have different causes and different con- 
sequences, and separate analyses of contact and cooperation give us infor- 
mation that will be useful in designing future immigrant surveys. But we 
also analysed how overall response, as opposed to non-response, depends 
on various characteristics of both sample persons and interviewers. 
Looking at overall response and non-response gives us information about 
the nature of the non-response bias in the data. For the estimations, we 
used multilevel models. 

The findings of the survey show differences between the three immi- 
grant groups investigated (Iran, Turkey and Pakistan) and this is at least 
partly related to the specific migration history of the immigrant groups. 
From Turkey and Pakistan, immigration started in the late 1960s and con- 
tinued in the beginning of the 1970s, where male immigrants from the two 
countries (together with male immigrants from the former Yugoslavia) 
came to Denmark to work as unskilled workers (“guest workers”) in the 
Danish manufacturing industry. In 1973, Denmark tightened its labour re- 
cruitment policy and introduced measures to reduce the influx of foreign- 
ers.” After 1973, only two major channels of legal immigration from non- 
Western countries to Denmark remained: family reunification and asylum 
(Bauer, Larsen & Matthiessen 2004). Many of the male guest workers 
stayed in Denmark and brought their families to the country. Moreover, 
many of the children of the Turkish and Pakistani guest workers have con- 
tinued to find their spouses in the country of origin (Schmidt & Jakobsen 
2000). 

In the second half of the 1980s and in the 1990s, immigration to 
Denmark increased. Family reunions (especially from Turkey and 
Pakistan) were still part of the picture, but the number of refugees seeking 
asylum also increased strongly. The immigration from Iran was part of an 
extensive increase in the number of refugees coming to Denmark in the 
second half of the 1980s together with refugees from Iraq, Lebanon and 
Sri Lanka; while in the 1990s many refugees came from Bosnia- 
Herzegovina and Somalia (Pedersen & Smith 2002). 


NON-RESPONSE AMONG IMMIGRANTS IN DENMARK 175 


After 2000, rules for family reunion and asylum were further tightened 
and the number of immigrants obtaining residence permits due to family 
reunion or asylum in Denmark decreased. Instead, immigration became do- 
minated by economic migrants (among others those from East European 
countries) and by students (Statistics Denmark 2011, Christensen 2010). 

The chapter is organised as follows. Section 8.2 describes our hypothe- 
ses about the relationship between various background characteristics of 
both sample persons and interviewers and the probability of contact and 
cooperation. Sections 8.3 and 8.4 describe the population sampling and the 
data collection, respectively. Section 8.5 presents the contact, cooperation 
and response rates in our survey, while section 8.6 presents the results of 
the estimations of the relationship between the background characteristics 
and probability of contact, cooperation and response. Finally, section 8.7 
concludes the chapter, discussing the main challenges and strategies 
adopted and their outcomes. 


8.2 Hypotheses linking characteristics of sample persons and 
interviewers with contact and cooperation 


The sample person characteristics most widely studied are socio-demo- 
graphic characteristics, such as age, marital status, household structure, ed- 
ucation, employment status, income and urbanisation (Groves & Couper 
1998).? Widely studied interviewer characteristics are gender, age and ex- 
perience as an interviewer (Campanelli & O’Muircheartaigh 1999). These 
factors may affect the contact rate and cooperation rate in different ways. 
This section describes our hypotheses linking characteristics of sample per- 
sons and interviewers with contact and cooperation (a summary of the hy- 
potheses is shown in Table 8.3 in section 8.6). 


Characteristics of sample persons and contacts 


The probability of making contact with the sample persons may be re- 
lated to the time they spend at home. Likewise, the time spent at home 
may be correlated with characteristics of the sample persons. For exam- 
ple, people who are busy with activities outside of the home (e.g. work 
and education) will be more difficult to contact (Abraham, Maitland & 
Bianchi 2006). This is in accordance with previous studies, which have 
shown that students and employed persons are more difficult to reach 
than those outside the labour force and those who are unemployed, just 
as people who work long hours are more difficult to reach than people 
who work part-time (Stoop 2004, Abraham et al. 2006). Time spent on 
activities outside the home, such as engagement in sports, may also be re- 
lated to age. Empirical studies typically find that contact rates are lowest 
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for young people (Groves & Couper 1998; Stoop 2004; Abraham et al. 
2006). 

The family or household structure may also matter for contact probabil- 
ity. The larger the number of adults in the household, the larger the proba- 
bility of establishing contact with someone who can then give information 
about the sample person (e.g. mobile phone number) or information about 
when the sample person is at home (Groves & Couper 1998). The implica- 
tion is that singles are more difficult to contact (Stoop 2004). In addition, 
the presence of young children may affect the contact probability if house- 
holds with young children have an adult caregiver at home more often than 
households without young children. Empirical studies thus show that hav- 
ing children in the household has a positive effect on the contact rate 
(Groves & Couper 1998; Stoop 2004). Furthermore, adult caregivers most 
often are women, implying that women are easier to contact than men, oth- 
er things being equal. 

Characteristics of the housing conditions and urbanisation may affect the 
interviewer’s probability of getting in contact with the sample person. 
Special security features, which may limit interviewer access, are typically 
more widespread in high-crime areas and in blocks of flats (i.e. in urban 
areas). Empirical studies have found a negative effect of urbanisation on 
the contact rate (Stoop 2004; Groves & Couper 1998). Living in an urban 
area may also be correlated with other individual characteristics that affect 
the probability of contact. For example, time spent outside the home may 
be higher in urban areas because more entertainment options are available; 
people in rural areas may be away from home more because of longer 
commutes; and there may be differences in employment rates and age 
structures between urban and rural areas. 

Thus, we expect that employment, a high level of education (through its 
positive effect on employment probability), and young age will have nega- 
tive effects on contact probability, because individuals with these character- 
istics are at home less often. Urbanisation is also expected to negatively in- 
fluence contact probability. On the other hand, being married and having 
children are expected to have positive effects on contact probability. 

In addition to the factors that are relevant both for Danes and immi- 
grants, we look at some specific immigrant factors: the greater the number 
of years since migration and having Danish citizenship are expected to 
have a negative effect on contact probability for two reasons. One is that 
both factors are positively related to economic assimilation (e.g. employ- 
ment probability and wage rate) (Ekberg 1994; Chiswick, Cohen & Zach 
1997; Husted, Nielsen, Rosholm & Smith 2001). The other is that both fac- 
tors may be positively related to participation in cultural and sports activ- 
ities in the local community, thereby suggesting lower contact probabilities 
after controlling for the employment situation. Another very important 
factor for the immigrants is language problems, which may make contact 
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more difficult. Not only communicating with the sample person, but also 
getting information about the sample person (e.g. from a spouse) will 
likely be more difficult. Hence, the effect of marriage on contact probabil- 
ity may be smaller for immigrants than for native Danes. 


Characteristics of sample persons and survey cooperation 


Our expectations regarding the relationship between sample person-specific 
characteristics and the probability of cooperation are based on the social 
isolation hypothesis (see Groves & Couper 1998). According to this hy- 
pothesis, social isolates are out of touch with mainstream culture, behaving 
in accordance either with sub-cultural norms or in rejection of the domi- 
nant norms. The hypothesis is that socially isolated persons will be less 
likely to cooperate with a survey request that represents the broader society 
(e.g. government agencies). 

Individuals with lower socio-economic status are expected to have lower 
cooperation rates, as they are likely to be alienated from central social in- 
stitutions. Conversely, individuals with higher socio-economic status may 
perceive themselves as occupying an important social place and conse- 
quently either have a higher sense of civic duty or recognise the value of 
survey data as a common good. However, the empirical evidence on the re- 
lationship between income and education on the one hand and cooperation 
rates on the other is mixed. For instance, some studies find a positive rela- 
tionship between the cooperation rate and education, while other studies 
find the opposite result (Groves & Couper 1998). 

The age of sample persons may affect the cooperation rate in different 
ways. However, the hypotheses concerning age mainly focus on the coop- 
eration rate for the elderly, who are not included in our survey (one hy- 
pothesis is that the elderly have lower cooperation rates because of disen- 
gagement; another hypothesis is that the elderly have a higher sense of 
civic duty, leading to higher cooperation). With respect to gender, most 
studies find either no gender effect on the cooperation rate or lower coop- 
eration rates for men. The explanations for the latter can also be related to 
the social isolation hypothesis, if women take more responsibility for social 
relations than men (Groves & Couper 1998). 

The social isolation hypothesis also predicts that household indicators af- 
fect cooperation: people living in single-person households are expected to 
have lower cooperation rates (tendency to social isolation), households 
with children tend to have higher cooperation rates (through schools and 
networks of friends) and those living in large blocks of flats have lower co- 
operation rates (less contact with neighbours, greater transience). Thus, 
sample persons in urban areas are likely to have lower cooperation rates 
than those in rural areas, because large blocks of flats are urban phenom- 
ena. Empirical studies show without exception that the presence of children 
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increases the cooperation rate, while the evidence with respect to single- 
person households is mixed (Groves & Couper 1998). 

Ethnicity (here measured by country of origin) may also effect the coop- 
eration rate, for example, if the ethnic minorities not do feel accepted by 
the majority society. Therefore, we expect that belonging to an ethnic mi- 
nority group will have a negative effect on the cooperation rate (after con- 
trolling for demographic and socio-economic factors which are correlated 
with ethnicity). 

Given the social isolation hypothesis, we expect being employed, having 
a high level of education, living in rural areas, being married and having 
children have positive effects on the cooperation rate. The number of years 
since migration and having Danish citizenship are also expected to have a 
positive effect on the cooperation rate, as these factors are related to the as- 
similation of immigrants into Danish society, while belonging to an ethnic 
minority group is expected to have a negative effect on the cooperation 
rate. 


Characteristics of interviewers and non-response 


No matter whether interviews are carried out by telephone or face-to-face, 
they involve both an interviewee and an interviewer, and it is natural to ex- 
pect that the interviewer may unintentionally affect the response rate. For 
instance, the interviewer’s expectations of and attitudes towards non-re- 
sponse may influence the response rate (Campanelli & O’Muircheartaigh 
1999). Although we have no information on the expectations and the atti- 
tudes of interviewers, socio-demographic characteristics of the interviewers 
and interviewers’ experience may affect their expectations and behaviour 
and therefore the response rate (Groves & Couper 1998). One study thus 
found that female interviewers are more likely than male interviewers to be 
perceived as friendly. However, little empirical evidence supports the view 
that female interviewers in general have higher response rates. 

Results concerning the age of the interviewers are also mixed 
(Campanelli & O’Muircheartaigh 1999). Interviewer experience is sup- 
posed to have a positive effect on the response rate (Groves & Couper 
1998). Pickery and Loosveldt (2002) find that both the chances of refusals 
and non-contacts are subject to interviewer effects and that the experience 
of the interviewer is particularly important. 


83 Population sampling of immigrants and Danes 
This section discusses the sampling of the survey analysed. The sampling 


was based on the Danish population register. In Denmark, every individual 
is assigned a personal identity number that is registered in the Danish 
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National Register of Persons (CPR). The CPR includes all individuals born 
in Denmark as well as individuals moving to Denmark and expecting to 
stay in the country for at least three months. Coverage of the CPR is ap- 
proximately 99.9 per cent. The existence of the CPR register makes it pos- 
sible to directly sample among individuals in the target population and, 
thus, the gross sample is representative of the targeted population. 

The sample was drawn in January 2006. The gross sample included a 
representative sample of approximately 1,000 immigrants from Iran, 1,000 
immigrants from Pakistan, and 1,000 immigrants from Turkey, as well as a 
representative sample of approximately 1,100 Danes. All persons included 
were 18-45 years of age and the immigrants included all came to Denmark 
before 1 January 2006. A total of 4,045 individuals were selected for the 
survey. 

The selection of immigrants from the three countries is based on the def- 
inition of immigrants in the Danish administrative registers. All individuals 
in the Danish registers are placed in three categories: Immigrants, descend- 
ants and native Danes. Immigrants are defined as persons who are foreign 
born and whose parents are foreign born or have a foreign citizenship. 
Descendants of immigrants are defined as persons born in Denmark, whose 
parents are foreign born or have a foreign citizenship. Native Danes 
(Danes for short) are defined as persons, who have at least one parent who 
is born in Denmark and is a Danish citizen (Pedersen 1991). In addition, 
all immigrants and descendants are linked with their country of origin — 
based on country of birth or mother’s country of birth. For the survey, only 
immigrants and Danes were sampled while descendants were excluded. To 
summarise, the sampling was done by linking information on personal 
identity numbers (CPR), immigrant status (immigrant or native Dane), 
country of origin for the immigrants (Iran, Pakistan and Turkey) and age 
(18-45 years). 

In addition to the sampling information, the Danish administrative regis- 
ters include information on a large range of issues. This information is 
avallable for all individuals in the gross sample, regardless of whether they 
responded to the survey. This provides us with a unique opportunity for an- 
alysing the distribution of various background characteristics for respond- 
ents compared to non-respondents. The information available from the ad- 
ministrative registers used in the analysis include gender, age, family situa- 
tion, region, citizenship, education obtained in Denmark, employment 
history in Denmark and years since migration. All of these background 
variables are from 2006, except for the variables on education and employ- 
ment, for which the latest available information is from 2005 and 2003, 
respectively. 
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8.4 Data collection 


Data collection was carried out in relation to a research project on the inte- 
gration of young immigrants into the Danish educational system and the la- 
bour market. The questionnaire included questions about family structure, 
years since migration, education (in both Denmark and the country of ori- 
gin), employment, working hours, job search, working experience, profi- 
ciency in Danish, social networks, housework, religion, and attitudes to- 
wards employment and gender roles. The length of the questionnaire was 
approximately forty minutes. 

The survey was announced by an introductory letter stating that an inter- 
viewer would contact the sample person by telephone or by visit to make 
an appointment for the interview. The immigrants received two letters: one 
in Danish and one in Farsi, Urdu or Turkish. The interview was primarily 
to be carried out in Danish, but the sample persons were informed that if 
this was too difficult, the interview could also be carried out in Farsi, Urdu 
or Turkish. For this purpose, the questionnaire was translated into the three 
languages. 

The interviewing was done by a professional interviewer from the sur- 
vey organisation (SFI Survey), either by telephone (CATI) or by face-to- 
face interviews (CAPI). The data collection was expected to be difficult 
and, hence, the interviewers were allowed a great deal of flexibility in try- 
ing to get in touch with the sample persons. The suggested starting point 
from the survey organisation was that the interviewer try to make contact 
by telephone at least six times at different hours of the day and on different 
days of the week. If the interview could not be done by telephone, the in- 
terviewer was to try to make an appointment for a visit. If telephone con- 
tact did not work, the interviewer was to visit the address at least three 
times. But as mentioned, the interviewers were allowed to make their own 
judgements and, for instance, start by making visits to the address 1f they 
believed this to be more feasible (and also if no telephone number could 
be found). Furthermore, if the interview could not be carried out in Danish 
the interviewer could pass on the sample person to an interviewer speaking 
the relevant language (the majority of interviewers were only Danish- 
speaking). Finally as the last alternative, the interviewer could leave a 
questionnaire at the address with a stamped addressed envelope for self- 
completion. 

A relatively long data-collection period was planned, from February un- 
til June 2006. However, by June the response rate was unacceptably low, 
especially among the immigrants from Pakistan. A main reason for the 
poor result was, among others, that language proved to be a larger obstacle 
than expected. In principle, the possibility to pass on interviews to bilin- 
gual interviewers would imply that no interviews were given up due to lan- 
guage problems. In reality, however, this created logistical problems as 
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well as queues. In addition, for some interviewers, data collection was so 
difficult that they more or less gave up. Therefore, the survey organisation 
decided to extend data collection until November 2006 and, furthermore, 
many sample persons were reassigned to other interviewers. 

A number of variables regarding the data collection are available from 
the survey organisation. This includes the mode of data collection. The ma- 
jority of the completed interviews were carried out by CATI — 93 per cent 
of the interviews for the Danes and between 67 and 76 per cent for the im- 
migrants. A large part of this difference is due to the language problem, as 
almost no interviews were carried out by telephone in languages other than 
Danish. Language problems were especially prevalent among the Turkish 
immigrants, with 43 per cent of the Turkish women and 37 per cent of the 
men interviewed by a Turkish-speaking interviewer. Among the immi- 
grants from Pakistan, the figures were almost as high. Some 38 per cent of 
the Pakistani women and 33 per cent of the men were interviewed by an 
Urdu-speaking interviewer. The figures for the Iranian immigrants were 
lower, with 27 per cent of the women and 21 per cent of the men inter- 
viewed in Farsi. The number of interviews done in a language other than 
Danish was substantially higher than expected and points to the necessity 
of having enough bilingual interviewers for such a data collection effort. 

In addition to the mode of data collection, we have information about 
reasons for non-response, if applicable, and number of contact tries. 
Furthermore, we have information about the interviewers from the survey 
organisation (SFI Survey), including their age, gender, experience as an in- 
terviewer (tenure in the survey organisation), and the number of interviews 
the interviewer was assigned. As mentioned, some of the sample persons 
were reassigned to other interviewers during the data-collection period. 
Unfortunately, in these cases we only have information about the last inter- 
viewer assigned to the sample person. Likewise, we do not know how 
many different interviewers each sample person was assigned to during the 
interviewing period. 


8.5 Response and non-response rates 


The response rates are much lower for the immigrant groups than for 
Danes (Table 8.1). This clearly demonstrates that the immigrant groups are 
more difficult to survey than the native population. However, the response 
rate also varies greatly across immigrant groups. Roughly speaking, the re- 
sponse rate is approximately 80 per cent for the Danes, 55 per cent for the 
Turks, 60 per cent for the Iranians, and 40 per cent for the Pakistanis. 
Gender, on the other hand, does not appear to be important. The only 
group with a significant gender difference in response rates is the Danes: 
women have a higher response rate than men (Table 8.1). 
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Table 8.1 Response and non-response (%) 


Denmark Iran Pakistan Turkey 


Men Women Men Women Men Women Men Women 


Response (completed and 76.9 82.0 627 649 414 423 55.0 55.3 
partial interviews) 


Non-response 23.1 18.0 3⁄4 35. 586 57.9 45.0 44.7 
Causes of non-response: 
Non-contact 8.2 4.1 16.8 14.4 32.8 29.7 16.1 12.6 
Refusals 13.6 12.8 13.4 14.4 16.9 17.1 22.9 25.5 
Other reasons 1.3 1.1 7.2 6.4 90 11.0 6.0 6.6 
Total 100 100 100 100 100 100 100 100 


Number of observations 536 562 573 390 510 475 516 483 


Source: Enrolment in Education and Jobs among Immigrants 2006 (www.dda.dk) 


Furthermore, we find marked differences across the four groups with re- 
spect to causes of non-response. We divide the non-response into three cat- 
egories: ‘non-contacts’, ‘refusals’ and ‘other reasons’. Generally, the share 
of non-contacts is much larger for the immigrants than for the Danes, as 
the interviewers experienced greater difficulties in making contact with im- 
migrants than with Danes. Pakistani immigrants were especially difficult to 
contact: the share of non-contacts for Pakistani immigrants is about twice 
the share of each of the other two immigrant groups. Refusals, however, 
appear to be greatest among the Turkish immigrants. Indeed, the share of 
refusals is relatively similar among Danes, Iranians and Pakistanis. 

The two non-response categories ‘refusals’ and ‘other reasons’ are bro- 
ken down into sub-categories, which give us more information about the 
causes of non-response.” The dominant ‘refusals’ sub-categories are ‘re- 
fusal due to lack of time’ (especially among men) and ‘refusal for other 
reasons’. Furthermore, for 4-5 per cent of the Turkish and Pakistani wom- 
en, their husbands refused on their behalf (whereas no woman refused on 
behalf of her husband). This situation poses a special problem for the inter- 
viewers, who have to convince another person to allow the interview be- 
fore they get the chance to convince the actual sample person. 

‘Other reasons’ includes sample persons who could not participate in the 
survey because they were ill, hospitalised, away from home, disabled or 
deceased, had moved out of the country or because of language problems. 
Non-response due to ‘other reasons’ is a greater problem among the immi- 
grants than among the Danes (Table 8.1). The immigrants are more likely 
to have either temporarily left the country or be unable to participate be- 
cause of language problems. Whereas language problems were seldom a 
reason for non-response among the immigrants from Iran and for the men 
from Pakistan and Turkey, about 5 per cent of the women from Pakistan 
and Turkey could not be interviewed because of language problems. In 
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principle, the language problem category should not exist because of the 
availability of bilingual interviewers, but as described earlier, the great 
need for bilingual interviewers created some logistical problems and 
queues. An additional problem might have been that some of the immi- 
grants only spoke a minority group language, such as Kurdish. 

To capture the probability of making contact with the sample persons 
and the willingness of the sample persons to participate in the survey, in 
this chapter we focus on the cooperation rate and the contact rate, together 
with the overall response rate. The cooperation rate is the share of inter- 
vlewed out of the total number of those who were interviewed plus those 
who refused to be interviewed. The contact rate is the share of sample per- 
sons that the interviewers made contact with out of the total number of 
sample persons. The response rate is defined as the number of completed 
and partially completed interviews as a share of the total number of sample 
persons. 

Table 8.2 shows the contact rate, cooperation rate and response rate for 
the four groups split into men and women. As mentioned, the contact rate 
is lower for immigrants than for the majority population, a result in accord- 
ance with other European studies (Feskens, Hox, Lensvelt-Mulders & 
Schmeets 2006). However, unlike Feskens et al. (2006), we do not find 
that the cooperation rate 1s higher for immigrants. On the contrary, we find 
that the cooperation rate is lower for immigrants than for Danes (although 
the cooperation rate for the Iranian immigrants is close to the Danish rate). 
Furthermore, notice that the cooperation rate is nearly the same for Turks 
and Pakistanis. Thus, the two groups do not differ much with respect to re- 
fusals given contact. Differences in the composition of the immigrant 
groups between Denmark and other European countries may explain why 
cooperation rates for immigrants are lower than for the native population 
in Denmark. 


Table 8.2 Response, contact, refusal and cooperation rates (%) 


Denmark Iran Pakistan Turkey 


Men Women Men Women Men Women Men Women 


Response rate 76.9 82.0 62.7 64.9 41.4 42.3 55.0 55.3 
Contact rate 91.8 95.9 83.2 85.6 67.2 70.3 83.9 87.4 
Cooperation rate 85.0 86.5 82.3 81.9 71.0 71.3 70.7 68.5 


Source: Enrolment in Education and Jobs among Immigrants, 2006 (www.dda.dk) 

Note: The calculations of the response, contact and cooperation rates are based on the 
standard definitions issued by the American Association for Public Opinion Research 
(AAPOR 2006). The cooperation rate we use is cooperation rate 4. Response rate = (I+P)/(I 
+P+R+NC+0); Contact rate = (I+P+R+0)/(I+P+R+0+NC); Cooperation rate = (I+P)/(I+P 
+R). I=Completed interview, P=Partial interview, R=Refusal and break-off, NC=Non-contact, 
O=Other 
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Overall, our results confirm that surveying immigrants is a challenge 
both for contact and for cooperation and that large variations exist across 
groups. That country-specific experiences should not be generalised to all 
immigrant groups is very clear; for example, Iranian immigrants are much 
more similar to Danes with respect to non-response than the Pakistani and 
Turkish immigrants. 


8.6 Analyses of contact, cooperation and response 


The previous section has shown that the two largest components of non-re- 
sponse are non-contacts and refusals; a result in accordance with most sur- 
veys. As mentioned in earlier, different types of non-response are likely to 
have different causes and different consequences. In the empirical analyses, 
we focus on the determinants of contact and cooperation to learn more 
about the causes of non-response and about the bias that results from non- 
response. In the model, we included sample person-specific variables as 
well as interviewer-specific variables. However, we found little impact 
from the interviewer-specific variables (age, gender, experience as an inter- 
viewer and number of interviews per interviewer) and therefore show only 
the results for the sample person-specific variables. 

The explanatory variables for the sample persons include information on 
gender, age, family situation (couple/single, no children/children) and re- 
gion (Copenhagen/other urban/rural). As mentioned, these register varia- 
bles are from 2006. Education is from 2005 and consists of the official du- 
ration of education obtained in Denmark. While there is some information 
in the registers about education obtained outside Denmark, the quality and 
coverage of this information is poor. Instead, we include a dummy variable 
for everybody without Danish education. Employment status is from 
November 2003 (the latest available information). For a minor share of the 
sample, no employment information is available — primarily because some 
immigrants were not in Denmark in 2003 but had immigrated or re-immi- 
grated in 2004 or 2005. 

For the immigrants, we include two specific variables: years since mi- 
gration and having Danish citizenship (having Danish citizenship/not hav- 
ing Danish citizenship). The latest information in the registers regarding 
time of immigration is from 2004; however, using other register informa- 
tion we can identify individuals who immigrated in 2005. Consequently, 
“years since immigration” is only unknown for about 1 per cent of the im- 
migrant sample. 

Our expectations of the impact of the sample person characteristics on 
contact probability and cooperation probability, respectively, are summar- 
ised in Table 8.3 and follow the discussion in section 8.2. Column 1 shows 
our expectations regarding contacts and column 2 shows those related to 
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Table 83 Overview of hypotheses linking characteristics of sample persons with 
contact, cooperation and response 


() (2) (3) 
Contact Cooperation Response 
Gender (men opposed to women) - (-) 
Age + > 
Couple (as opposed to single) + + + 
Children + + + 
Urban (as opposed to rural) - 
Education + ° 
Employment + 2 
Years since migration + 2 
Danish citizenship + 2 
> 


Country of origin (origin in Turkey, Iran or 2 
Pakistan opposed to origin in Denmark) 


Source: Authors' own elaboration 


cooperation. Overall response is a weighted average of contact and cooper- 
ation. If the bias in contact and cooperation point in different directions, 
the results may be that the overall response is unbiased. However, if a fac- 
tor has a negative impact on both contact and cooperation, we would ex- 
pect the factor to also have a negative impact on response rate (column 3 
in Table 8.3). 

The immigrants and the Danes differ with respect to the distribution of 
the characteristics in Table 8.3 and these differences in characteristics may 
be part of the explanation of the between-group differences in the contact 
and cooperation rates.? The immigrants from Pakistan and Turkey have 
lower employment rates and fewer years of education than Danes. 
According to the hypotheses this should imply higher contact rates and 
lower cooperation rates for immigrants than for Danes (other things being 
equal). Furthermore, a greater proportion of the immigrants from Turkey 
and Pakistan live in urban areas (especially the Pakistanis), which should 
imply lower contact and cooperation rates. They are also married and have 
children in greater proportion than Danes, which should imply higher con- 
tact and cooperation rates for immigrants. The Iranian immigrants also have 
lower employment rates than Danes, but are very similar to the Danes with 
respect to years of education. Furthermore, they are single and without chil- 
dren in greater proportion than Danes. However, as this survey demon- 
strates, the differences in composition of the four groups do not explain the 
large differences in contact and cooperation rates between the groups. 


Results 


The estimation results for the pooled sample of immigrants and Danes are 
shown in Table 8.4. Apparently, to a great extent there are different 
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characteristics that affect contact and cooperation. Furthermore, the charac- 
teristics that affect either cooperation or contact also affect overall response 
(one exception is gender). 

The demographic variables affect the probability of contact and re- 
sponse, but not the probability of cooperation. We find that contact is more 
likely for the age group 18-29 years compared to the left-out category, 30- 
39 years. Although young people are typically considered to be more diffi- 
cult to contact, this assumption is not confirmed here and we find that 
being young is associated with a higher response rate. The contact proba- 
bility is higher for women than men. In accordance with our hypotheses 
we also find that individuals who live in couples and have children are eas- 
ier to contact and have a higher response rate. 

Several studies have shown that urbanisation has a negative effect on 
the probability of contacting the sample persons. We likewise find that the 
probability of contact is significantly lower in Copenhagen than in other 
areas. Yet, no significant difference exists between the response in other ur- 
ban and rural areas. Besides a negative influence on the probability of 


Table 8.4 Probability of contact, cooperation and response, pooled sample 


Contact Cooperation Response 
Women 0.2834** -0.0005 0.1024 
Age group: 18-29 0.5069%** 0.1875 0.273 5% 
Age group: 40-45 0.2181 0.0802 0.0989 
Single re -0.0635 dd 
Children 0.1218 
Copenhagen -1.3747%** -0.5909*** 
Rural area 0.2322 -0.0428 
Danish education (in years) 0.0487 0.0713** 
No Danish education 0.5198 0.4558 0.3736 
Non-employed -0.1844 -0.1864*** 
Employment unknown -0.1211 0.9187 
Years since migration 0.0052 0.0005 
Years since migration unknown 0.4263 2.1114+** 
Danish citizen -0.0356 
Iran 0.0068 
Pakistan -0.1864 
Turkey -0.4941** 
Interviewer-variables Yes 
Constant 0.7908 
Number of level 1 units 3,098 
Number of level 2 units 53 
Log L -1,218 -1,495 -2,172 


Sources: Enrolment in Education and Jobs among Immigrants, 2006 (www.dda.dk) and 
register data from Statistics Denmark 

Note: The model applied is a logistic random intercept model. *significant at 10%, ** 
significant at 5%, *** significant at 1% 
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contact, living in Copenhagen also has a negative effect on the probability 
of cooperation. Thus, as expected, living in a metropolitan area has a 
strong negative effect on the response. 

We do not find any effect of “years since migration” and socio-economic 
status — measured by education and employment variables — on the contact 
probability. Thus our expectation that employed people, for example, 
would be more difficult to contact because they spend more hours away 
from home than unemployed people, is not confirmed. We do, however, 
find a positive effect on contact probability of having Danish citizenship, a 
factor that is related to economic assimilation. On the other hand, years of 
education and employment have — in agreement with the social isolation 
hypothesis — a positive influence on cooperation. Those with a Danish citi- 
zenship, several years of education and employment have a relative high 
response rate. 

The country dummies show that, after controlling for all other factors, 
the contact probability is lower for immigrants from Pakistan and Iran than 
for Danes.* The contact rate is lower in all the three immigrant groups than 
for Danes (see Table 8.2). However, for the immigrants from Turkey this 
result is attributable to the characteristics included in the model. On the 
other hand, the probability of cooperation is lower for immigrants from 
Turkey than for the other groups. That immigrants from some countries 
have a lower cooperation rate is in accordance with the isolation hypothe- 
sis. The probability of response is lower for all three immigrant groups 
than for Danes.” 

We can summarise the bias in overall response as follows: being young, 
living in couples and having children, more years of education, being em- 
ployed and having Danish citizenship increase the probability of response, 
while living in Copenhagen has a negative influence on the probability of 
response. Furthermore, the country dummies for Iran, Turkey and Pakistan 
are also significant — having controlled for all other variables, immigrants 
from the three countries exhibit lower response rates than Danes. 


8.7 Challenges and strategies 


In this survey, both contact rates and cooperation rates are lower for immi- 
grants than for Danes, leading to a significantly lower overall response 
rate. Furthermore, we found important differences between groups: the im- 
migrants from Pakistan are especially difficult to contact, and cooperation 
is low among immigrants from both Turkey and Pakistan. In addition, lan- 
guage is extremely important, as a particularly large share of women from 
Turkey and Pakistan could not be interviewed in Danish. 

To learn more about the causes of non-response, we analysed the deter- 
minants of the probability of contact and the probability of cooperation, 
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respectively. To learn more about the bias resulting from non-response, we 
analysed the determinants of overall non-response. We found that the char- 
acteristics of the sample persons are important for both the contact rate and 
the cooperation rate, and that different factors highly affect contact and co- 
operation. Gender, age and family structure significantly affect contact, 
while education and employment — in accordance with the social isolation 
hypothesis — affect cooperation. Living in Copenhagen has a negative ef- 
fect on both contact and cooperation. The characteristics of sample persons 
that significantly affect either contact or cooperation also significantly af- 
fect overall response. Furthermore, we found that the lower probability of 
response among immigrants when compared to Danes persists after con- 
trolling for all the other variables. Thus, the results clearly show bias in the 
overall response rate with respect to characteristics of the sample persons, 
along with a ‘country-of-origin’ factor beyond the factors that we can in- 
clude in the model. It should be noted, however, that non-response will al- 
so be influenced by other characteristics of the sample persons than those 
included in the register data. Such variables, for example, proficiency in 
Danish or having a network including native Danes, could explain the 
“country of origin” factor. 

Finally, another result is the lack of impact of the interviewer-specific 
variables. Unquestionably, the individual interviewer is important for the 
data-collecting process, but this impact cannot be measured by the type of 
very aggregate information that is available about the interviewers in this 
survey. It is worth noting that the gender of the interviewer did not affect 
the probability of achieving a successful interview. Female interviewers 
did not have more luck with female sample persons than male interviewers. 
However, the interviewers faced another problem: the husbands of 4-5 per 
cent of the Turkish and Pakistani women included in the sample refused 
participation on their behalf. 

The analysis clearly points to the need to tailor surveys directed at immi- 
grant groups to avoid response bias. Furthermore, the effort to reduce bias 
in overall response should focus on the contact phase as well as the persua- 
sion phase. 

When it comes to the contact problems it remains a puzzle why the im- 
migrants from Pakistan were especially difficult to contact. Interviews were 
completed for only 40 per cent of the persons of Pakistan origin in the 
sample, and the proportion of non-contacts was twice that for the other 
groups of immigrants. The vast majority of the population of Pakistani ori- 
gin in Denmark is living in the Copenhagen area. It was also more difficult 
for the interviewers to make contact with the selected persons among na- 
tive Danes living in Copenhagen. But the difficulties were vastly greater 
among the Pakistanis than among the native Danes, and they cannot be ex- 
plained by differences in individual characteristics as demonstrated above. 
A solution might be to earmark resources to allow interviewers additional 
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time to trace people, for example, by asking neighbours or others. Another 
solution might be to prolong the period for data collection in order to make 
contact with persons who have travelled to their country of origin. 

When it comes to cooperation problems it remains a puzzle why a rela- 
tively large number of immigrants from Pakistan as well as from Turkey 
refuse to participate in surveys. One countermove might be to rethink how 
the purpose of a survey is presented in the introduction letter sent out be- 
fore first contact. 

It is certainly also important to address language problems. We have no 
information about knowledge of Danish among the sample persons, and 
even if we did try to find solutions we cannot rule out that language was a 
hindrance to cooperation. As described above, a large number of immi- 
grants, especially women from Turkey and Pakistan, were interviewed by a 
bilingual interviewer. However, some interviews were never completed due 
to a lack of trained bilingual interviewers. It is certainly a challenge for the 
survey organisation to recruit and train bilingual interviewers with knowl- 
edge of the languages of the major immigrant groups. This problem might 
be complicated by the fact that bilingual interviewers (according to the in- 
terviewers) sometimes have more difficulty in achieving interviews with 
persons of other immigrant groups than would native Danish interviewers. 
In addition, interviewers from the same country of origin may have diffi- 
culties in gaining the trust of the sample persons — again, according to 
some interviewers. In small, local immigrant communities, where every- 
body knows everybody, respondents may fear gossip in their community 
and might rather prefer a native Danish interviewer. More knowledge about 
these issues is important in order to increase response rates in future 
surveys. 


Notes 


1 The name of the survey referred to here is “Indvandrere i uddannelse og job, 2006” 
(Enrolment in education and jobs among immigrants, 2006). The survey data are available 
from the Danish Data Archive (www.dda.dk). The register data from Statistics Denmark 
are, however, restricted access. 

2 Since the 1960s, Denmark has experienced positive net immigration as opposed to the pe- 
riod between World War II and the 1960s, during which Denmark experienced net emi- 
gration (Bauer et al. 2004). 

3 Groves and Couper (1998) categorise urbanisation under the category ‘societal 
environment’. 

4 Other things being equal, we expect a person’s network to be larger the longer the person 
has lived in the local community. 

5 The distribution on all sub-categories appears in Deding et al. (2008). 

6 We apply logistic random multi-level models — more precisely a logistic random intercept 
model to analyse the determinants of contacts, cooperation and response. For the estima- 
tions, we use the Stata Program GLLAMM (see e.g. Rabe-Hesketh et al. 2005). 
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7 The mean values of all the variables are shown in Deding et al. (2008). 

8 That the lower contact rate for the immigrant groups is not solely attributable to a higher 
concentration of immigrants in Copenhagen is confirmed by separate estimation for 
Copenhagen (see Deding et al. 2008). 

9 With the sample divided into four subgroups, the findings are highly similar to the pooled 
sample. Which variables are significant differs from group to group (in many cases due to 
the small sample size). However, there are a few differences compared to the pooled sam- 
ple (see Deding et al. 2008). 
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PART Ill 


INCLUDING IMMIGRANTS IN 
GENERAL POPULATION SOCIAL 
SURVEYS 


9 Immigration and general population surveys 
in Spain: The CIS surveys 


Mónica Méndez, Marisa Ferreras and María Cuesta! 


9.1 Introduction 


The arrival of significant immigration flows to Spain in the last two deca- 
des has been identified by many scholars as the factor with the greatest po- 
tential to trigger social and political transformations in Spain (Cebolla & 
González-Ferrer 2008; Gonzalez-Enriquez 2009; Reher & Silvestre 2009). 

Two traits characterise the way in which Spain has become a country of 
immigration: intensity and speed. This has created challenges in a number 
of areas, such as the design of public policies, provision of public services 
and absorption of a new labour force by the economy. A third trait worth 
mentioning is that Spain, in addition to being the destination of migrants 
from less developed countries seeking a job and better living conditions, 
has also been chosen as a (semi) permanent home by a considerable num- 
ber of people from developed countries, primarily from the European 
Union. These are mostly retired persons and they are concentrated on the 
Mediterranean coast and the Canary Islands. 

Immigration has also had an impact in the field of social and political 
surveys. As the presence of immigrants in Spain has grown, so too has in- 
terest in knowing the attitudes and opinions of Spaniards regarding this 
new phenomenon. More recently, there has been a rising demand for more 
knowledge about the socio-demographic profile, values and attitudes of the 
immigrant population. 

Unlike other chapters in this book, which focus on surveys specifically 
addressed to immigrants and ethnic minorities, the main objective of this 
chapter is to analyse the extent to which the growing presence of a foreign 
or foreign-born population is affecting the work of a large institution which 
carries out a range of different types of surveys and has research goals not 
related to immigration. It reviews how the Centro de Investigaciones 
Sociológicas (CIS, Sociological Research Centre) has faced the implica- 
tions of the presence of significant numbers of immigrants in general popu- 
lation surveys. CIS is a public institution whose main task for more than 
thirty years has been to carry out public opinion surveys. Every year it car- 
ries out more than thirty surveys on different topics. Among these, there 
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are regular surveys such as monthly public opinion barometers which have 
a series of fixed questions that are repeated, evaluating the political and 
economic situation and identifying what is perceived to be the three most 
important problems Spain faces. Every three months a survey evaluating 
the government's performance is carried out, rating ministers, evaluating 
the main party in the opposition as well as asking questions on voting 
intentions. 

This chapter will first look at how the target population of many of the 
surveys carried out by CIS has changed from including only those holding 
Spanish nationality to including all of the resident population (i.e., includ- 
ing foreigners). It will also look at the outcome of this change in terms of 
the presence of the foreign/immigrant population among the survey re- 
spondents. In addition, a detailed analysis is presented of the response rates 
of both Spaniards and the foreign population in the Spanish version of the 
International Social Survey Programme survey on religion, which was car- 
ried out in 2008. From that experience, the different potential barriers to 
the participation of immigrants in surveys are examined. 


9.2 Immigrant population and CIS surveys 


The topic of immigration has been present in the work of CIS since the 
very beginning of this phenomenon in Spain, especially in monitoring the 
perceptions and attitudes of Spaniards towards this issue. The first surveys 
on the topic date back to the early 1990s, when the percentage of immi- 
grants in the total population was still very low. Since then, many surveys 
have been devoted to measuring and analysing the attitudes of Spaniards 
towards immigration, particularly after 2000, when it became a salient is- 
sue in the public agenda. 

CIS was also an early pioneer in carrying out surveys specifically ad- 
dressed to the immigrant/foreign population in Spain. In 1996 it did a sur- 
vey of immigrants who were going through the process of regularisation of 
their residence in Spain. A year later it did a survey of Moroccans and 
Senegalese living in Spain within the project ‘Push and Pull Factors of 
International Migration” led by NIDI/Eurostat. The main aim of that initia- 
tive was to study the push and pull factors determining international migra- 
tion flows, surveying individuals in both sending and receiving countries.” 

More recently, in 2007 the interest of CIS in surveying immigrant popu- 
lations continued with a survey carried out in Arganda del Rey, a munici- 
pality on the outskirts of Madrid characterised by a high percentage of for- 
eign population, particularly Romanians. The aim of the survey was to ob- 
tain data in order to analyse the attitudes and behaviour of Romanians 
regarding electoral participation (in the 2007 local elections) as well as 
their associational patterns. The design of the survey included a control 
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sample of the rest of the nationalities residing in the municipality (mainly 
Spaniards), so as to gain greater insight in the interpretation and analysis 
of the data obtained for the Romanian residents. A final example of the 
work of CIS in studying immigrant populations is a qualitative study based 
on focus groups, whose aim was to get a panorama of the attitudes and dis- 
courses of immigrant groups regarding their participation in different as- 
pects of economic, social and political life in Spain (Colectivo IOÉ 2010). 

In conclusion, CIS was a pioneer in Spain in including immigration as a 
major concern in its surveys, both those that measure attitudes of 
Spaniards towards immigrants and immigration and also, albeit to a lesser 
extent, in carrying out surveys to find out the profile, attitudes and values 
of (certain groups within) the immigrant population. However, as this 
chapter will show, it is only in the last decade that we have really begun to 
see the impact of the presence of immigrants on the way that CIS carries 
out its main surveys addressed to the general population. 


Definition of the target population in CIS general population surveys 


In general, “the Spanish population” is the universe of the surveys done by 
CIS since its creation in 1977, and of its predecessor, Instituto de Opinión 
Publica (Public Opinion Institute). This has been the definition used in vir- 
tually all of the methodological reports of the surveys done since the 
1970s. This does not appear to be an intentional decision, made to adjust 
the population of interest to people holding Spanish nationality. Initially, 
the percentage of foreign population residing in Spain was so small that in 
practical terms referring to the “Spanish population” was equivalent to the 
“resident population in Spain”. 

The increasing presence of a foreign/immigrant population during the 
past decade changed the implications of the term ‘Spanish population’, 
meaning people who hold Spanish nationality. Thus, a survey addressed to 
the ‘Spanish population’, following the traditional definition used by CIS 
in its surveys, leaves out of its universe approximately 10 per cent of the 
population living in Spain (i.e. those that do not have Spanish nationality) 
as a result of a sort of inertia in the application of a longstanding definition 
of the population of interest. As a matter of fact, not all immigrants from 
other countries are excluded using this definition, but only the ones that 
have not obtained Spanish nationality. 

The questioning of this definition came first in the surveys about sub- 
jects related to public policies and the provision of public services. As a re- 
sult, in 2003, for the first time the population universe for the Health 
Barometer was the resident population in Spain and not just individuals 
holding Spanish nationality. This is a survey that CIS has been carrying 
out since the mid-1990s, commissioned by the Ministry of Health. The aim 
is to obtain information about the performance of health policies and the 
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degree of user satisfaction with the public and private health service. Since 
2003 the target group of this survey has been the resident population. 

In addition to this survey, from year to year the number of surveys ad- 
dressed to the resident population has increased, as well as the variety of 
topics covered. Surveys have addressed such issues as the evaluation of 
public services, sexual health and attitudes towards the treatment of termi- 
nally ill patients. Thus, an important change in recent years has been that 
the decision on the target population is no longer “automatic” or brought 
about by inertia, but is rather the result of a choice about whether to en- 
large the survey to the resident population in order to include non- 
nationals. 

However, in spite of this change, the target population for a good part of 
the surveys done by CIS is still the Spanish population. This is the case for 
the monthly barometers described above, as well as for most of the elector- 
al studies that are carried out when regional and general elections are held. 
This is due to the fact that in these elections voting rights are reserved for 
those holding Spanish nationality. 

In addition to paying more attention to the foreign/immigrant population 
in the definition of the universe for each survey, some effort has also been 
made in the adaptation of questionnaires. Although this was previously 
done sporadically, since 2008 all questionnaires have included at least one 
question regarding nationality (when a Spanish nationality sample is 
sought it is the first question so as to identify eligible interviewees). The 
question includes the option ‘double nationality’ in the responses, given 
the high number of people from Latin American countries who may have 
both Spanish nationality and nationality of their country of origin. 
Sometimes this question is complemented by others, such as when Spanish 
nationality was obtained. In surveys addressed to the resident population, 
there is a question about the country of birth so as to be able to distinguish 
between native born and immigrants. However, the inclusion of both ques- 
tions in nearly all surveys is fairly recent, so in many of the surveys al- 
ready carried out on the resident population there is usually only the ques- 
tion about nationality. This is the reason why in this chapter, when survey 
data from the CIS is used, we sometimes compare Spaniards to foreigners, 
and when possible, we compare native born with foreign born. 

In addition to what has been done regarding questionnaires, some work 
has begun to check the extent to which some of the fixed socio-demo- 
graphic questions initially designed in a context of high cultural homoge- 
neity — such as the ones about education level or religious affiliation and 
practice — can be adapted to better reflect the new diversity brought about 
by immigration. 
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The presence of foreign population among the respondents of CIS general 
population surveys 


Changing the definition of the target population from Spanish to resident 
population is a necessary but not a sufficient condition for the foreign pop- 
ulation to be properly represented in public opinion surveys. This section 
looks at the presence of a foreign population among the respondents of all 
the surveys that CIS has addressed to the resident population in Spain since 
2003. Figure 9.1 shows the percentage of the final respondents that are for- 
eigners in surveys addressed to the resident population aged 18 years and 
older. The black columns show the percentage of foreigners in the total 
population aged 18 and older (the target population of the surveys), as pro- 
vided by the Spanish National Statistics Institute on the basis of the popu- 
lation register (“Padrón”), which includes foreigners that are registered (re- 
gardless of whether they are legally authorised to reside in Spain). 

All the surveys shown in the figure were done face-to-face, and most of 
the samples, except for two (studies 2671 and 2776), have a similar design 
and size (2,500-2,600 individuals). They are three-stage stratified samples. 
Strata are obtained by crossing two classification criteria: the Autonomous 


Figure 9.1 Percentage of foreigners in the total population interviewed in CIS 
surveys addressed to the resident population (2003-2009) 
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Source: Authors' own elaboration with data from CIS Data Bank and from INE 
Notes: The number of surveys are provided so that they can be located in the 
online CIS Data Bank. Black columns show the percentage of foreigners in the 
total population aged 18 and older (the target population of the surveys). Dark 
grey columns show percentage of foreigners among survey respondents. Light 
grey columns are surveys that used full probability samples. 
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Community (the name given to Spanish regions) and the size of habitat 
(seven categories: 1-2,000 inhabitants, 2,001-10,000, 10,001-50,000, 
50,001-100,000, 100,001-400,000, 400,001-1,000,000, and 1,000,000 and 
over).? Primary (municipalities) and secondary sampling units (census sec- 
tions) are drawn randomly and proportionally to the population living in 
each stratum, whereas the final selection of the individual to interview is 
made following random routes in the census sections included in the sam- 
ple in order to fulfil a particular quota of individuals classified by sex and 
age. The two exceptions mentioned are full probability samples, since the 
final selection of individuals is made randomly from the ‘Padrón’. 

The main conclusion drawn from Figure 9.1 is clear: foreigners are 
under-represented in CIS surveys. The percentage of foreigners as shown 
in the light grey columns is always less than the percentage they represent 
in the total resident population in Spain 18 years and older. There are sig- 
nificant fluctuations in the extent to which this under-representation occurs 
that seem to bear no relation to the topic of the survey, given that some of 
the health barometers obtain a better representation in their final samples 
of foreigners than others. 

The year 2009 stands out in particular because of the relatively constant 
and comparatively poor representation of the foreign population in the sur- 
veys done that year. It is not easy to find an explanation for this in the 
methodology used to carry out the surveys, which is the same as was used 
in previous years. It could be that during 2009, when the economic crisis 
was already hitting Spain badly, there was a greater movement of immi- 
grants, either back to their countries or to other countries. If this was the 
case, the figures from the Population Register would be a less adequate 
yardstick with which to measure the representativeness of survey respond- 
ents than in other years when immigration was growing. These movements 
are always difficult to know from the ‘Padrón’, given that people tend to 
register but not to ‘de-register’ when they move to a different country. The 
new rule that registration in the “Padrón” expires if not confirmed every 
two years could improve the accuracy of this register to measure the stock 
of foreign residents in Spain (see chapter 4 in this volume). 

The under-representation of foreigners also occurs in the surveys of 
young people (15-29 years old) that are shown in Figure 9.2: the percen- 
tages of foreigners (in lighter grey) is lower than the percentage they con- 
stitute of the total population for those ages residing in Spain (black col- 
umns). In this case this under-representation seems to have gotten worse 
beginning in the second half of 2008. 

These results fit with what we can expect on the basis of the literature 
on response rates of foreigners and immigrants. Hypotheses that might ex- 
plain this have already been discussed in other chapters of this book (see 
chapters 7 and 8). Among the most important are greater difficulties locat- 
ing immigrant/foreign population than is the case with autochthonous 
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Figure 9.2 Percentage of foreigners in the total population interviewed in CIS 
surveys addressed to the young resident population 15-29 years of age 
(2003-2009) 


Source: Authors' own elaboration with data from CIS Data Bank and from INE 
Notes: The number of surveys are provided so that they can be located in the 
online CIS Data Bank. Black columns show the percentage of foreigners in the 
young resident population aged 15-29 (the target population of the surveys). 
Dark grey columns show percentage of foreigners among survey respondents. 


populations, language difficulties, cultural barriers and lack of trust (partic- 
ularly in the case of unauthorised immigrants). 

When interpreting the data from CIS surveys it has to be kept in mind 
that so far no changes have been made in the way the fieldwork is carried 
out to adapt to the presence of foreigners. Interviews are carried out face- 
to-face in Spanish, and no extra effort is made to hire interviewers with a 
foreign or immigrant background (though some of them do have such a 
background). 

Table 9.1 contains the main nationalities present among the respondents 
of all of the face-to-face surveys done by CIS in 2008 which were ad- 
dressed to the resident population. Figure 9.3 shows the main nationalities 
among the foreign population and the percentages they represent out of the 
total resident population according to the “Padrón”. The nationalities with 
the highest presence among survey respondents usually match those that 
are also the most prevalent among the resident population. Thus, 
Ecuadorians, Romanians and Moroccans are usually the most numerous 
non-national groups of respondents, and they are also the three main for- 
eign national groups present in Spain according to the official figures. The 
greater presence of Ecuadorians among survey respondents is probably due 
to them being native Spanish speakers, but the fact that Romanians and 
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Table 9.1 Main foreign nationalities present among respondents in the surveys 
carried out by CIS of the resident population in Spain (aged 18 years and 
older) in 2008 (percentages in total final sample) 


Survey main topic Sample size Percentage of foreigners Main nationalities 
out of total respondents 
Religion 2,500 10.1 Ecuadorian (1.79%) 


Romanian (1.5%) 
Moroccan (0.9%) 
Colombian (0.8%) 


Health barometer 2,600 9.1 Romanian (1.6%) 

(1st wave) Ecuadorian (0.8%) 
Peruvian (0.8%) 
Moroccan (0.6%) 


Quality of public 4,550 6.2 Romanian (0.9%) 
services Ecuadorian (0.9%) 
Colombian (0.8%) 
Moroccan (0.8%) 


Health barometer 2,600 9.1 Ecuadorian (1.3%) 
(2nd wave) Moroccan (0.9%) 
Romanian (0.8%) 
Colombian (0.5%) 


Religion 4,000 7.6 Ecuadorian (1.3%) 
(ISSP module) (2,373 complete Moroccan (1%) 
interviews) Romanian (0.9%) 


Colombian (0.6%) 


Health barometer 2,600 10.1 Ecuadorian (1.1%) 

(3rd wave) Romanian (1%) 
Moroccan (0.8%) 
Bolivian (0.7%) 


National survey on 10,000 10.5 Ecuadorian (1.2%) 

sexual health Romanian (1.1%) 
Colombian (1%) 
Argentinian (0.9%) 


Source: Authors’ own elaboration with data from CIS Data Bank 


Moroccans are not does not prevent them from being relatively well repre- 
sented in the final samples. The only exception is the National Survey on 
Sexual Health, for which the three main nationalities present among non- 
national respondents are Ecuadorians, Romanians and Colombians. 

The main explanation for this exception might be related to the topic of 
the survey, sexual health, which is likely to generate a greater reluctance to 
participate among Moroccans than among other nationalities. Unfortunately, 
the data regarding the reasons for refusal to participate did not include the 
nationality of the person who refused, and therefore it is impossible to verify 
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Figure 9.3 Main nationalities among foreigners residing in Spain (percentages over 
the total population of 18 years old) 
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that hypothesis. The fact that there was no gender matching between inter- 
viewee and interviewer, so that men would only be interviewed by men and 
women with women, was probably detrimental for the participation of 
Moroccan women. According to the pilot test conducted, gender matching 
was not considered necessary for the target population as a whole, but it 
might have made certain groups of people/nationalities feel more at ease 
about participating in the survey. 

Another result that is worth pointing out is the practical absence of re- 
spondents from the United Kingdom in the surveys analysed. It is the nation- 
ality with the greatest under-representation. Individuals of UK origin are the 
fourth most numerous group but have extremely low participation in sur- 
veys. For example, only 0.1 per cent of the respondents of the National 
Survey on Sexual Health were born in the United Kingdom (i.e. 7 respond- 
ents out of 9,849), instead of the roughly 0.8 per cent they should have been 
if nationals from the United Kingdom had been accurately represented in the 
survey. The under-representation of this group is probably related to the fact 
that this is generally a population that is geographically concentrated, mainly 
on the Mediterranean coast and the Canary Islands. This makes it less likely 
that they will be present in the samples. Secondly, there is their insufficient 
command of Spanish to consider. A large proportion of this population is re- 
tired people living in areas where they can socialise and get by without learn- 
ing much Spanish. For example, in 2009 nearly half of the UK nationals that 
resided in Spain were over 55 years old, while in the case of Ecuadorians, 
Romanians and Moroccans this figure was less than 4 per cent.* 
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All of the surveys included in the tables and figures of this section, ex- 
cept two (“es2671” and “es2776” in the lightest grey columns in Figure 
9.1), use probability sampling to select census sections, but the final selec- 
tion of the individual to be interviewed is made on the basis of sex and 
age quotas. This could mean that the lower propensity of foreigners to par- 
ticipate in surveys (due, e.g. to longer working hours, difficulties with lan- 
guage) could result in their under-representation, so they would be “substi- 
tuted” with Spaniards. However, as the next section will show, the detailed 
analysis of one of the two surveys mentioned above, which were done with 
a full probability sample, prove that this under-representation cannot be at- 
tributed to the quotas. In both the 2007 ISSP “role of government” survey 
and the 2008 “religion” survey, the percentage of foreigners in total re- 
spondents was lower than their actual presence in the probability sample, 
and lower than in the surveys with sex and age quotas that were carried 
out more or less during the same period. 


9.3 Example of a survey with a nominal probability sample: 
ISSP 2008 religion survey 


This section presents an analysis of the outcomes of the fieldwork of the 
Spanish ISSP survey on religion, which was carried out in the autumn of 
2008. As was pointed out above, unlike other surveys done by CIS, the 
sample of this survey is probabilistic, so the selection of individuals to in- 
terview is made randomly among those residents in each census section in- 
cluded in the sample who are aged 18 years and older (and registered in 
the population register). This is the same sampling frame used by the 
Spanish National Statistics Institute for the ENI survey (see chapter 3) and 
also the one used in other important social attitudes surveys, such as the 
Spanish edition of the European Social Survey. 

Table 9.2 shows the profile of the sample used and of the respondents of 
the survey, according to sex and age group. The foreign population 
amounts to 12.1 per cent of the sample, more or less in line with the 11.5 
per cent that it represented in the total resident population aged 18 and old- 
er, according to the official figures. The profile of the foreign population 
selected corresponds to what is already known about this population. That 
is, more men than women are included and the age structure tends to be 
younger than that of the Spanish population, although there are important 
differences according to nationality. 

The percentage of foreigners in the total number of respondents goes 
down to 7.6 per cent, four points less than in the initial sample. Thus, con- 
sistent with the outcomes presented in other chapters in this book, the re- 
sponse rate among foreigners was considerably lower than among 
Spaniards. While 62.4 per cent of Spaniards included in the sample were 
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interviewed, only 37 per cent of the foreigners were. Table 9.2 shows im- 
portant variations in response rate according to nationality, age group and, 
to a lesser extent, sex. 

Regarding nationality, all foreign nationalities are under-represented 
among the final respondents compared to the initial sample; this is espe- 
cially the case with individuals from Western European countries. There 
are also important differences in response rate among Latin Americans, 
both comparing men and women and different age groups. 

Table 9.3 shows a more detailed account of the outcome of each of the 
cases included in the initial sample. An analysis of these outcomes permits 
the identification of the main barriers that foreigners/immigrants face in 
participating in surveys, especially general population surveys that do not 
tend to have the resources to make special efforts to reach certain groups 
of the population. Section 9.4 examines these barriers in more detail. 


9.4 Immigrant/foreign population in general population 
surveys: Barriers to participation 


Difficulties in locating immigrants and foreign population 


The main reason for the lower response rate of foreigners included in the 
2008 ISSP survey is the difficulty of locating (and contacting) a large per- 
centage of them. As shown in Table 9.3, among all of the reasons for not 
carrying out an interview, this is by far the most important one, among 
both foreigners and Spaniards. 

These are especially high non-contact rates compared to other surveys 
that use a similar methodology, such as the Spanish edition of the 
European Social Survey. The main differences between the two can be ex- 
plained because the highest level of non-contact in Table 9.3 refers to the 
cases in which no contact has been established with the person in the sam- 
ple, even if someone from the same dwelling has been contacted. There 
are other rules for calculating response rates and criteria for eligibility that 
are different: for example, having moved to another country is considered 
a cause for ineligibility in the European Social Survey, but not under the 
rules of the AAPOR (American Association of Public Opinion Research), 
which are the ones followed by the ISSP. 

However, not living at the address under which the person selected in 
the sample is registered in the ‘Padrón’ stands out as the main reason for 
inability to establish contact with the selected person, given that in most 
cases the person living at the selected address could not or did not want to 
provide a new address to find the person in the sample. This occurred with 
33 per cent of the foreign persons included in the sample, while that figure 
is just 6 per cent in the case of Spaniards. The higher residential mobility 
of foreigners is well known; the results of the 2007 Spanish immigrant 
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Table 9.3 Detailed outcome of the fieldwork (% of total sample), 2008 ISSP survey 


on religion 
Outcome Spaniards Foreigners Detailed outcome Spaniards Foreigners 
Completed 62.4 37.2 62.4 37.2 
interviews 
Refusal 13.3 3.7 Refusal of the selected 10.0 2.3 
person 
Refusal to open door 3.1 1.2 
Non-contact 19.5 51.9 Empty dwellings 5.4 5.8 
Selected person absent all the 4.1 4.3 
times the interviewer visited 
the dwelling* 
Selected person absent 2.3 5.0 
during the whole fieldwork 
period 
Selected person does not live 6.0 33.1 
in the address under which 
he/she is registered 
(sampling frame) 
Other reasons for non- 1.7 3.7 
contact (difficulty to access 
the dwelling, not a private 
dwelling, contact letter 
returned, etc.) 
Other non- 4.8 7.4 Sickness/death of selected 3.3 0.4 
responses person 
Language impediments 0.0 5.6 
Other reasons 1.7 1.4 
Total 100 100 100 100 
Number of 3,516 484 3,516 484 


observations 


Source: Authors’ own elaboration with data from contact sheets of the survey ISSP Religion 


(2008) (CIS study 2776) 


Note: All the data refer to the information gathered at the last call the interviewers did with 
each person in the sample 


survey (ENI) showed that nearly three out of four foreigners had changed 
their address at least once since arriving in Spain (INE 2007: 95). Other re- 
search points in the same direction and highlights the great difference with 
Spaniards, who are characterised by a relatively low residential mobility 
(Recaño 2002; Recaño & Domingo 2006). 

The remaining reasons for not establishing contact have a very low inci- 
dence both among Spaniards and foreigners. The only one with a higher 
incidence among foreigners is “absent during the fieldwork period”, which 
was applicable to cases in which interviewers were informed, most often 
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by a third person, that the person selected in the sample would not be at 
that address for the whole time the fieldwork lasted. These could either be 
cases where the person selected had in fact moved to a different address, 
but the informant did not want to say so, or they had gone to their home 
countries for a visit — such visits often extending a long time (the fieldwork 
lasted 45 days altogether). 


Survey cooperation 


The refusal rate among foreigners in the 2008 ISSP study in Spain is 3.5 
per cent, a figure substantially lower than that found among Spaniards (13 
per cent). These figures include both the refusal of the person selected in 
the sample and that of another person in the household (e.g. refusing to 
open the door to the interviewer, or refusing on behalf of the person in the 
sample). To some extent, this is linked to the greater non-contact rate 
among foreigners. In other words, logically speaking, if refusal takes place, 
it is after contact has been established, which is when the person in the 
sample has to decide whether to participate in the survey. 

However, there are other indications that point to a greater disposition 
on the part of the foreign population to participate in surveys in compari- 
son to native Spaniards, as is the case in other European countries 
(Feskens, Hox, Lensvelt-Mulders & Schmeets 2006). Some of these indica- 
tors can be found in a survey done by CIS in 2009 addressed to the resi- 
dent population aged 18 and older, which included a number of questions 
regarding respondents” attitudes towards surveys, their willingness to par- 
ticipate in them, and their preference regarding mode of administration. 
One of these questions asked respondents why they had agreed to partici- 


Table 9.4 Main reasons to participate in the survey 


Immigrants Spanish Total 

(born in Spain) sample 

To help the interviewer 45.0* 49.9 49.4 
To see what it was like to be interviewed 18.3* 12.7 13.2 
Because he/she wanted to give his/her opinion 16.4 17.1 17.0 
He/she always participates in surveys when asked 5.7 5.0 5.0 
Interested in the topic of the survey 5.0 3.9 4.0 
Because it was done by CIS 2.0* 5.9 5.6 
Other reasons 3.7 3.0 3.1 
Don't know 2.4 1.9 2.0 
No answer 1.5 0.7 0.7 
Total 100 100 100 
Number of observations 458 4,412 4,870 


Source: Authors” own elaboration with data from the survey Satisfaction of User of 
Telecommunication Services in Spain (CIS 2797, 2009) 
* Differences between immigrants and Spanish born statistically significant at the 0.05 level 
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pate in that survey in particular (results shown in Table 9.4). 

There are no large differences between the two populations: for both the 
Spanish born and immigrants the most prevalent response to why they de- 
cided to take part in the survey was because they wanted to help the inter- 
viewer. Among the foreign born the second most cited reason was curios- 
ity, wanting to know what it was like to be interviewed. This makes sense 
if we take into account that the percentage of people for whom this was 
the first time to participate in a survey was much greater among the foreign 
born than among the Spanish born respondents. For the remaining possible 
answers (interest in the topic of the survey, because it was done by CIS 
and others) the responses were similar among immigrants and Spanish 
born respondents. 

Regarding the possibility of repeating the experience and participating 
again in a (face-to-face) survey, most respondents said they would agree to 
be interviewed on a different occasion (Table 9.5). The percentage of im- 
migrants who said they would do so is slightly higher than that of Spanish 
born (67 and 62 per cent, respectively). 

Among the reasons provided by those respondents who would not par- 
ticipate again in a survey (Table 9.6), lack of time stands out among the 
immigrant population, while more Spanish-born respondents give lack of 
trust in surveys as a reason. 


Table 9.5 Hypothetical response of the person interviewed to a request that they 
participate in a face-to-face survey (at his or her home) 


Immigrants Spanish Total 
(born in Spain) sample 
Almost sure | would agree to take part 31.4* 25.0* 25.6 
| would probably agree to take part 35.4 37.1 36.9 
| would probably not agree to take part 5.0 6.7 6.6 
Almost sure | would not agree to take part 4.6 5.9 5.8 
It depends (not in the questionnaire, volunteered 14.6 14.2 14.3 
answer) 
It depends on who does the survey (volunteered 1.3 2.2 2.1 
answer) 
It would depend on the topic (volunteered answer) 2.8 3.7 3.6 
Don’t know 3.9 4.5 4.5 
No answer 0.9 0.6 0.6 
Total 100 100 100 
Number of observations 458 4,412 4,870 


Source: Authors' own elaboration with data from the survey Satisfaction of User of 
Telecommunication Services in Spain (CIS 2797, 2009) 

Note: These three options characterised as 'volunteered answers' were not provided in the 
questionnaire and show cards, but were recorded if they were spontaneously mentioned by 
respondents 

* Differences between immigrants and Spanish born statistically significant at the 0.05 level 
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Table 9.6 Reasons for declining future participation in a hypothetical face-to-face 


survey 
Immigrants Spanish Total 
(born in Spain) 

Because | didn’t like the experience this time 4.5 3.4 3.5 
Because | never have time 38.6 28.9 29.6 
Because | don’t trust surveys 6.8 10.9 10.6 
Because | don’t think my data are treated 15.9 14.5 14.6 
anonymously/ | don’t believe the privacy of my data is 
respected 
Because | think surveys are useless 9.1 10.6 10.5 
Other reasons 4.5 9.1 8.8 
Don’t know 4.5 3.0 3.2 
No answer 15.9 19.5 19.3 
Total 100 100 100 
Number of observations 44 558 602 


Source: Authors’ own elaboration with data from the survey Satisfaction of user of 
telecommunication services in Spain (CIS 2797, 2009) 


Table 9.7 contains other interesting information that can help us to 
understand why the immigrant population seems to be more inclined to 
participate in surveys. Some 58 per cent of the Spanish born who partici- 
pated in this survey said that they had already participated in a survey prior 
to this one (and most people more than once). This percentage goes down 
to 40 per cent in the case of the foreign-born population. This might mean 
that the immigrant population is less affected by ‘survey fatigue’, an ex- 
planation that is frequently used to explain declining response rates in most 
Western countries. 


Table 9.7 Classification of the surveyed population with respect to their previous 
participation in surveys (any mode of administration) 


Immigrants Spanish Total 
(born in Spain) 


Had already taken part in other surveys 28.2* 45.2 43.6 
(more than once) 

Had already taken part in other survey (once) 11.6 13.0 12.9 
First time to take part in a survey 60.3 41.3 43.1 
No answer 0.0 0.5 0.5 
Total 100 100 100 
Number of observations 458 4,412 4,870 


Source: Authors’ own elaboration with data from the survey Satisfaction of User of 
Telecommunication Services in Spain (CIS 2797, 2009) 
* Differences between immigrants and Spanish born statistically significant at the 0.05 level 
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Language difficulties 


In the 2008 Spanish ISSP survey, language difficulties, that is, not speak- 
ing Spanish well enough, was the reason why 27 people could not partici- 
pate in the survey. This amounts to 5.6 per cent of the total number of for- 
eigners included in the sample. Of those cases, eight were British and eight 
Moroccan; these two groups constitute 60 per cent of those who could not 
complete the interview due to problems with the language. The difficulties 
in locating a large percentage of the foreign population included in the 
sample does not allow us to know what percentage of that said population, 
if located, would have language difficulties. 

However, the information gathered by CIS in other surveys leads to the 
tentative conclusion that in general terms, language does not seem to be a 
significant impediment to participation of immigrants and foreigners in sur- 
veys. At least, 1t does not seem to be very important in explaining the 
under-representation of this population in surveys. 

These are the main conclusions of a recent experiment carried out to de- 
termine the extent to which language barriers would be an impediment to 
changing the current target population of CIS barometers from the Spanish 
population to the resident population (i.e. including foreigners). In the bar- 
ometers done from February to December 2009 information was obtained 
about the people who were discarded because they did not have Spanish 
nationality regarding their command of the Spanish language. The inter- 
viewers were also asked to assess the language skills of the contacted per- 
son. The nationality, sex and age group of each contacted person was also 
registered. 

With that information we can see how many people would not be able 
to do the interview even if they were part of the population of interest. 
These would include, first of all, people unable to speak Spanish at all, and 
1t seems reasonable to include in this group those who “speak little 
Spanish’. However, we do consider that individuals who speak Spanish 
“more or less’ would be able to do an interview. Table 9.8 follows these 
criteria and classifies all the foreign people contacted while doing the field- 
work for the monthly barometers. The percentage of those who would not 
be able to do the interview anyway, because of having an insufficient 
knowledge of Spanish, comprises 30 per cent of all the foreign persons 
contacted, while almost half of them speak Spanish well enough to be able 
to do the interview without difficulties. 

We may wonder about the prevailing criterion, whether that of the inter- 
viewee or that of the interviewer, to determine whether someone can or 
cannot do an interview. However, the data show that this is not a worrying 
issue, given that the assessment of the potential interviewee and that of the 
interviewer coincide in most cases (82 per cent). In around half of the 
cases in which the evaluations differ, the assessment of the contacted 
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Table 9.8 Classification of foreign persons contacted in the fieldwork of the 
barometers of the CIS (February 2ooo-January 2oto) based on their 
command of Spanish, percentage of total foreigners contacted 


Assessed by Assessed by 
potential interviewer 
interviewee 

(1) Does not speak it at all 5.6 6.0 
(2) Speaks it a little bit 23.3 22.9 
(1)+(2) 28.9 28.9 
(3) Speaks it more or less well 20.9 18.5 
(4) Speaks it fluently 13.9 13.9 
(5) Speaks it as if it was his/her mother tongue 5.7 8.1 
(6) It is her/his mother tongue 29.6 29 
(4)+(5)+(6) 49.2 51 
Don't know/No answer 1.0 1.6 
Total (1+2+3+4+5+6) 100 100 
Number of observations 5,099 5,099 


Source: Authors’ own elaboration from data from the Fieldwork Unit, CIS. Monthly CIS 
barometers February-December 2009 


person of their use of Spanish is greater than that of the interviewer, while 
in the other half the reverse occurs. 

The results reveal important and expected differences in the command 
of Spanish by the foreign population living in Spain, depending on various 
criteria, such as nationality of origin and age group (Table 9.9). 

Following the same criterion mentioned above, according to which only 
those individuals who at least speak Spanish more or less would be able to 
do the interview, from the data in Table 9.9 we can single out three scenar- 
los. There is a first group of people formed by nationals from Latin 
American countries. Most of them have Spanish as their mother tongue 
and would have no problem being interviewed if they were part of the tar- 
get population. This does not mean that there are no occasions where some 
of these people would find it difficult to understand certain expressions or 
particular words, due to the different varieties of Spanish that exist 
throughout the world. But obviously such differences are minor and much 
less of a hindrance than the language difficulties faced by other foreigners. 
In a second group we find mostly Moroccans and Romanians, the two for- 
eign nationalities most present in Spain. They are divided into two groups: 
those who do not speak Spanish or at least not well enough to do an inter- 
view and those who speak it fluently (or as a native speaker does). The 
third scenario is that of nationals from other countries, the majority of 
whom speak very little or no Spanish at all. They would not be able to do 
the interviews even if they were part of the target population. In this group 
we find a wide range of nationalities, such as the British and the Chinese. 
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Summing up, it is clear that doing interviews only in Spanish is not opti- 
mal, as it excludes that part of the foreign population living in Spain that 
has a different profile from those who are able to do the interview in 
Spanish. However, given the economic resources and the time needed to 
translate questionnaires into the relevant languages and the need for a net- 
work of interviewers that is able to administer the questionnaires in these 
languages, the benefits obtained from such a huge investment would also 
be limited. Data shown in this section reveal that, while being important, 
difficulty with language is a minor problem when trying to adequately rep- 
resent the foreign/immigrant population in surveys. 


9.5 Concluding remarks 


This chapter has analysed the way in which CIS, an institution whose main 
mission is to carry out surveys to monitor Spanish public opinion on vari- 
ous political and social issues, is dealing with the new scenario created by 
immigration. At first, immigration was considered a phenomenon that de- 
served to be studied and analysed as one among other topics that were ana- 
lysed through surveys; it was not thought to be an issue that would affect 
the way CIS carried out its surveys. 

The gradual increase in the percentage of immigrants and foreigners, 
now constituting more than 10 per cent of the total population, and the 
foreseeable permanence of this demographic change has slowly changed 
this initial consideration. The main effect of this has been the gradual but 
steady increase of the number of surveys whose target population has be- 
come the resident population, rather than the Spanish population, as was 
previously the case. Some surveys have been addressed specifically to the 
non-national population. However, so far the fundamentals of CIS survey 
methodology have not been adapted to this new reality: there is no transla- 
tion of questionnaires into other languages, no particular effort to adapt the 
network of interviewers, and the fieldwork organisation remains the same. 

The chapter has also looked at how foreign residents are represented in 
the surveys in which they are part of the target population. In all of them 
the percentage of foreigners among respondents is less than what it should 
be, according to population data. As in other countries, immigrants and for- 
eigners are under-represented in general population surveys in Spain. In 
the surveys done with probability samples but with final selection of re- 
spondents on the basis of sex and age quota sampling, this under-represen- 
tation exists but is slightly lower than that found in nominal probability 
samples, such as the ones used for the ISSP surveys in 2007 and 2008. 
The inability to locate around a third of the foreigners included in the sam- 
ple at the addresses where they were registered in the “Padrón” (the popula- 
tion register used as the sampling frame) is the main explanation for the 
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under-representation of foreigners. Using a different type of probability 
sampling with a different selection of the individuals to be interviewed, for 
example, through Kish tables or the birthday date selection method, would 
perhaps help to solve that problem and achieve a better representation of 
foreigners and immigrants in the final sample of respondents. 

Language is not a fundamental barrier to attaining a good representation 
of foreigners in surveys, although it is important to keep in mind that a 
considerable proportion of the Moroccan, Romanian and Chinese popula- 
tion do not speak Spanish well enough to be able to do a survey interview. 
Language is also a problem among the nationals of Western European 
countries, in particular the United Kingdom and Germany, the majority of 
whom do not speak Spanish; hence, the presence of nationals from these 
countries among respondents is very low. In any case, the population for 
many of the foreign nationalities residing in Spain does not reach the 
threshold adopted in the European Social Survey, which requires that if a 
language is spoken by at least 5 per cent of the target population then the 
questionnaire must be translated. 

Usually, when analysing the potential impact of not being able to prop- 
erly represent the foreign population in surveys, most attention is given to 
people who have migrated to Spain to work, rather than to those who mi- 
grate when they have reached the age of retirement (which is mostly the 
case among UK or German nationals living in Spain). Obviously in terms 
of the technical bias, the relevance of this under-representation is equal re- 
gardless of country of origin. However, the social and political consequen- 
ces of this misrepresentation might be different according to nationality. 

The main challenge in order to obtain a better representation of foreign 
residents in probability surveys if nominal/addressed samples continue to 
be used is to improve the contact rates, given that this is the main reason 
for not carrying out interviews with foreigners included in the samples. 
Using other types of probability samples that do not require looking for 
specific people at specific addresses would certainly help, but the example 
of the surveys done with quota samples show that this is not enough, and 
further, that a combination of measures would probably be the best way to 
achieve a better representation: slight adaptation of fieldwork procedures 
(adapting the timing of interviews to better suit the foreign population), 
providing training to interviewers to be able to adapt to different types of 
non-national respondents and other fairly modest similar changes would al- 
so help. 

The number of general population studies that will use residents rather 
than Spaniards as the target population is likely to increase in the future. 
The decision to do so also depends on issues such as the comparability of 
time series which are especially valuable if they are long, as is the case 
with the monthly CIS barometers, which offer time series for particular 
questions that are more than thirty years old. The comparability of data 
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would be affected if the target populations of the surveys were changed. 
Having good information on the changes brought about by the enlargement 
of the definition of the target population would help to facilitate the transi- 
tion from one target population to the new one. In any case, having data 
on the first experiences and applying those analyses to reflect on the best 
ways to obtain a good representation of foreigners and immigrants is a fun- 
damental step to greater inclusion of these groups in general population 
surveys. 


Notes 


1 The views expressed in this chapter are those of the authors and do not necessarily reflect 
those of CIS. 

2 All the details of the survey design can be obtained in the country report (Arango, 

García-Pardo, Laseca & Martínez 2000). The Spanish data are stored in the CIS Databank 

(CIS Study 2232, 1997). 

Ceuta and Melilla, the two Spanish enclaves in the north of Africa, are excluded. 

4 Data from the “Padrón” (Instituto Nacional de Estadística, INE). 
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10 An evaluation of Spanish questions on the 
2006 and 2008 US General Social Surveys 


Tom W. Smith 


10.1 Introduction 


How much does the picture we obtain about a given society depend on 
whether we conduct interviews in one or two languages? This chapter pro- 
vides a first attempt to answer this question through the analysis of the ex- 
perience of the US General Social Survey (GSS) and its incorporation of 
Spanish as a second interviewing language. 

The GSS is part of the National Data Program for the Social Sciences, a 
social indicators infrastructure and data-diffusion programme. Its basic pur- 
poses are three: 1) to gather data on US society in order to monitor and ex- 
plain trends and constants in attitudes, behaviours and attributes and to ex- 
amine the structure and functioning of society in general as well as the role 
of various subgroups; 2) to compare the United States to other societies to 
place American society in a comparative perspective and to develop cross- 
national models of human society; and 3) to make high-quality data easily 
accessible to scholars, students and others with minimal cost and waiting. 

These purposes are accomplished by the regular collection and distribu- 
tion of the GSS (www.gss.norc.org) and its allied surveys in the 
International Social Survey Programme (ISSP) (www.issp.org). The GSS 
has conducted 27 national, in-person, full-probability samples of adults liv- 
ing in households in the United States between 1972 and 2008. A total of 
53,034 respondents have been interviewed (Davis, Smith, & Marsden 
2009). 

In 2006, GSS secured support from the National Science Foundation to 
add Spanish to its standard, English-language version. With this expansion 
the GSS target population became adults living in US households and able 
to do an interview in either English or Spanish. According to the US 
Census, Hispanics were 12.5 per cent of the population in 2000 and an es- 
timated 15.5 per cent in 2010. The Census also reports that 41 per cent of 
Hispanics five years of age and older do not speak English “very well”. 

The main aim of this chapter is to show that the use of questionnaires in 
Spanish brought about the participation in the GSS of immigrants who 
could not have participated if the survey had only been conducted in 
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English, or that, at least, would not have done so in the same way. Thus it 
shows how an important decision in the survey design may have an impact 
on the survey results. The chapter describes the process by which the 
Spanish-language version of the GSS was developed. It then looks at the 
different language-use and ability groups in which Hispanics are distrib- 
uted. The third section looks at the changes brought about in the coverage 
of the Hispanic population and in the total target population of the GSS. 
Section four presents evidence of language problems in the translations. 
The final section presents implications for trend analyses using the GSS. 


10.2 Translation of questionnaires 


Translation of the questionnaire from English into Spanish utilised the fol- 
lowing procedures. First, using the committee-translation approach, transla- 
tions were done by Research Support Services (RSS) (Schoua-Glusberg 
2006). Under the RSS committee-translation approach, three translators si- 
multaneously and independently translated the questions. They then met 
and compared their versions. Under Schoua-Glusberg's direction they rec- 
onciled disagreements and settled on a collaborative translation that pro- 
vided the best functional equivalence to the original GSS wordings. In ad- 
dition, the committee strove to come up with a Spanish version that was 
equally suitable for the various Hispanic sub-populations in the United 
States (e.g. Puerto Rican, Mexican, South American). 

Second, RSS’s translations were reviewed by a bilingual NORC staffer, 
and in collaboration with RSS changes were made in various items. 
Finally, the revised Spanish translation was reviewed by the GSS director. 
He and the bilingual NORC staffers discussed various points and the final 
Spanish version was adopted. 


10.3 The 2006 GSS: Some data on the coverage of Hispanic 
population 


Altogether, on versions 1-6 of the 2006 GSS 427 Hispanics were inter- 
viewed.' Of these 220 were interviewed in English and 227 in Spanish. No 
non-Hispanics were interviewed in Spanish.” 

Language use/ability of respondents was divided into four categories: 1) 
did the interview in English, not interviewable in Spanish, 2) did the inter- 
view in English, interviewable in Spanish, 3) did the interview in Spanish, 
interviewable in English, and 4) did the interview in Spanish, not inter- 
viewable in English. Thus, there are the English monolinguals, two groups 
of bilinguals (depending on which language was used) and the Spanish 
monolinguals. These four categories were operationalised in four different 
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ways. Language of interview was a fixed attribute, so the different imple- 
mentations were based on which of two measures of English ability was 
used to measure bilingualism among the Spanish interviewees and where 
the self-assessment of Spanish ability was cut among English interviewees. 
The first measure of English ability among Spanish interviewees was an 
assessment by the interviewers. The interviewers answered the following: 


Before 2006 the GSS was only administered in English. Those 
without enough English to do the interview were excluded as out- 
of-scope, language problems. In 2006 a Spanish version of the GSS 
was added. If there had been no Spanish version available in 2006, 
could this respondent have been interviewed in English or do you 
think she or he would have been excluded as a language problem? 


Could have been interviewed in English 1 
Would have been excluded as a language problem 2 


The second measure of English ability among Spanish interviewees was 
a self-assessment by respondents: 


If this interview had only been available in English, would you... 


Have been able to do the interview easily in English, 1 
Have been able to do the interview with difficulty, or 2 
Not have been able to do the interview? 3 


Interviewers judged that 14.5 per cent of Spanish interviewees could 
have done the survey in English and that 85.5 per cent would have been 
excluded as language problems. Among Spanish respondents 9.9 per cent 
said they could have easily done the survey in English, 21.5 per cent could 
have done it with difficulty, and 68.6 per cent could not have done it.* 
There was a high degree of agreement between these two independent 
measures. Some 94.6 per cent of those who said that doing an interview in 
English would have been easy were rated by interviewers as interviewable 
in English. Conversely, 99.6 per cent of those who said they could not 
have done an English interview were similarly judged by interviewers. Of 
those who indicated that they could have done the interview with ‘diffi- 
culty’, 22.5 per cent were rated as interviewable in English by inter- 
viewers, and 77.5 per cent were rated as not interviewable. This largely 
negative evaluation is consistent with the respondents’ self-assessment that 
an English interview would have been difficult. 

On versions 1-6, respondents interviewed in English were asked the fol- 
lowing questions: 
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Can you speak a language other than English? 
What other language(s) do you speak? 
How well do you speak that language? 


Very well 1 
Well 2 
Not well 3 
Poorly/Hardly at all 4 


Spanish speakers were identified through these items. The first lan- 
guage-use/ability scale classifies as English monolinguals those indicating 
no Spanish, English/Spanish as those with any Spanish ability, Spanish/ 
English as those judged by interviewers as able to have done the interview 
in English, and Spanish monolinguals as those deemed by interviewers as 
unable to have done an English interview. The second language-use/ability 
scale divides English cases the same as the first, but uses respondents” self- 
assessments of English ability and includes those in the easy and difficult 
group in the bilingual group and only those unable to have done an 
English interview as Spanish monolinguals. The third language-use/ability 
scale has English monolinguals as those with no Spanish plus those who 
spoke Spanish hardly at all and English/Spanish bilinguals as those speak- 
ing Spanish better than poorly. As in the first scale, Spanish respondents 
were divided according to the interviewers’ evaluation into bilinguals and 
Spanish monolinguals. The fourth language-use/ability scale has English 
monolinguals and English/Spanish bilinguals classified as in the third scale 
and Spanish/English bilinguals and Spanish monolinguals according to re- 
spondent self-assessment as in the second scale. 

The distribution of Hispanic respondents across these four scales on ver- 
sions 1-6 of 2006 GSS is shown in Table 10.1. Thus, the second scale min- 
imises the size of the monolingual categories and the third scale maximises 
the monolinguals. The bilingual groups are closest in size in the fourth 
scale and furthest apart in the first scale. In effect, the language-use/ability 
scales form a measure of general language assimilation for Hispanics that 
should associate with other measures of assimilation. 

As expected, the addition of a Spanish-language version appreciably ex- 
panded the coverage of Hispanics. As Table 10.2 shows, by various meas- 
ures the coverage of Hispanics rose from 9.1-10.4 per cent in 2004 to 
14.4-16.8 per cent in 2006. In addition, the profile of Hispanics changed in 
certain ways.* 


10.4 Differences in socio-demographic profile of Hispanics 


As Table 10.3 indicates, with Spanish language added, the profile of 
Hispanics changes: they are less assimilated (fewer born in the United 
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Table 10.1 Distribution of Hispanic respondents across the four language use/ability 
scales 


English only English Spanish Spanish English Spanish only All (N) 


First scale 42 178 29 178 427 
Second scale 42 178 66 141 427 
Third scale 60 160 29 178 427 
Fourth scale 60 160 66 141 427 


Source: General Social Survey, 2006 


Table 10.2 Trends in Hispanic coverage on GSS, 1996-2006 


1996 1998 2000 2002 2004 2006 Prob. 


Main ethnicity (ETHNIC) 74 7.2 8.3 8.8 104 16.8  .000 
Any Hispanic (ETH1, ETH2, ETH3) 6.7 6.5 7.6 8.6 92 154 .000 
Hispanic. vs. Other (HISPANIC) _ _ 8.1 8.1 9.1 14.5 .000 


Source: General Social Survey (1996-2oo6) 


Table 10.3 Changes in the profile of Hispanics (%), GSS 2000-2006 


2000 2002 2004 2006 Prob. 
Male 43.9 43.4 48.3 48.3 456 
Under 30 30.7 32.2 40.2 28.1 .033 
Married 53.9 50.9 49.4 52.6 181 
No children 27.2 29.0 34.4 26.2 .000 
Working full-time 71.0 56.3 62.7 57.0 131 
College degree 13.8 16.4 17.8 12.0 .000 
Catholic 70.7 65.1 62.6 69.5 .095 
Both parents from the US 37.2 32.8 38.4 25.3 .000 
Born in the US 53.2 60.0 69.5 36.9 .000 
Lived in US at age 16 70.3 69.5 84.0 49.3 .000 


Source: General Social Survey (2ooo-2oo6) 


States, living in the United States at age 16, or with parents born in the 
United States), less educated, younger and fewer have no children. There 
were no differences on gender, marital status, labour-force status and 
religion. 

As data in Table 10.4 show (see appendix), Hispanics differ appreciably 
in their socio-demographic profile by language use/ability. First, there is a 
strong assimilation gradient with living in the United States at age 16, 
being born in the United States, and having parents born in the United 
States falling from a high among English monolinguals to lower levels 
among English bilingual and then Spanish bilinguals to the lowest level 
among Spanish monolinguals. Second, the percentage of Catholic rose 
from a low among the English monolinguals to a high among Spanish 
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bilinguals and Spanish monolinguals. This is also probably an assimilation 
effect. Third, living in a non-entry state had a complex and unexpected re- 
lationship to language use/ability.° 

As expected, living outside the entry states was highest among the 
English monolinguals. But it was then second highest for the Spanish 
monolinguals, third highest for the English bilinguals and very low and last 
for the Spanish bilinguals. Thus, this does not follow a simple assimilation 
model of geographic dispersion. Fourth, education is highest among the 
English monolinguals and lowest among the Spanish monolinguals. Fifth, 
full-time, labour-force participation is highest among bilinguals and lower 
among both monolingual groups. Sixth, the English monolinguals have the 
largest proportion under 30 and the Spanish monolinguals the smallest 
share. Seventh, the English monolinguals have the fewest married people 
and Spanish bilinguals and monolinguals have the highest proportion mar- 
ried. Eighth, having no children is greatest among the English monolin- 
guals and lowest among the Spanish monolinguals. Finally, gender and ru- 
ral-urban residence do not vary by language use/ability. 


10.5 Differences in attitudes, behaviours and other non- 
demographic variables 


What difference does the use of Spanish make in the results of the survey? 
Does the inclusion of immigrants who could not have taken part in the sur- 
vey because of a language problem change the final picture obtained? 
Table 10.3 has already shown that in 2006 the profile of the Hispanic pop- 
ulation in the survey changed considerably. In this section attention is fo- 
cused on the impact of the use of a Spanish language questionnaire in the 
overall results of the surveys, regarding the portrait it provides of attitudes, 
behaviours and other non-demographic variables in the United States. In 
other words, can we know what would have happened if Spanish had not 
been used? 

In order to do so, Table 10.5 (in the appendix) examines 35 non-demo- 
graphic variables by language use/ability. These items were selected to rep- 
resent the full range of non-demographics in the 2006 GSS. Examples 
were examined from all major substantive scales (e.g. abortion, Stouffer 
civil liberties, confidence in institutions) and a wide range of response 
scales (e.g. agree/disagree, numerical, unbalanced trichotomies). First ex- 
amined were the differences across the four language-use/ability levels for 
each of the four classifications. Overall, there was a high number of statis- 
tically significant differences (109 out of 140). These were about equally 
common across the four classification methods (first 25/35; second 27/35; 
third 28/35; and fourth 29/35). 
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Next, differences between just the two bilingual groups were considered. 
Given the smaller samples and the more limited group differentiation, con- 
siderably fewer statistically significant associations appeared (29 out of 
140). Statistically significant differences were more common for classifica- 
tion 2 and 4 (ten each) than for 1 and 3 (respectively, five and four). This 
is probably mainly due to classifications 2 and 4 counting more people as 
bilinguals (226-244) than classifications 1 and 3 (189-207) did. 

As with both the trends and language-use/ability group differences 
shown for the demographic variables, this variation across language-use/ 
ability groups indicates some notable shifts in the profile of Hispanics. 
Table 10.5 presents the full results, but examples include that 1) support 
for capital punishment declines from English monolinguals to Spanish 
monolinguals, 2) support for more foreign aid is greater for Spanish-lan- 
guage cases than English-language cases, 3) support for science spending 
declines from English to Spanish monolinguals, 4) middle class self-identi- 
fication falls from English to Spanish monolinguals, 5) financial satisfac- 
tion is lowest among Spanish monolinguals, 6) job satisfaction is highest 
among the bilinguals, 6) self-rated health is highest among the English 
monolinguals and lowest among the Spanish monolinguals, 7) finding ho- 
mosexual activity “always wrong” increases from English monolinguals to 
the Spanish-language respondents, 8) having close friends who are either 
Black or White is greatest among the English monolinguals, 9) Democratic 
party identification is highest among the bilinguals, and 10) presidential 
voting declines from among the English monolinguals to the Spanish 
monolinguals. 


10.6 Language effects 
Attitudes and behavioural questions 


Besides indicating shifts in the profile of Hispanics, data in Table 10.5 can 
also be used to assess possible language effects. In looking for possible 
language effects two conditions were required: 1) that there was statisti- 
cally significant variation across the language-use/ability continuum and 2) 
that there was a statistically significant difference between the English and 
Spanish bilingual groups. Next, two additional factors were considered: 1) 
on how many of the four classification approaches were the first two con- 
ditions met and 2) did the pattern of difference across language-use/ability 
groups show the largest difference between the two bilingual groups. The 
pattern most suggestive of a language effect would be little or no differ- 
ence between the English monolinguals and English bilinguals, a large dif- 
ference between the English and Spanish bilinguals, and little or no differ- 
ence between the Spanish bilinguals and Spanish monolinguals. That repre- 
sents an inter-language difference with no evidence of intra-language 
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difference across the language-use/ability groups. By contrast a pattern 
with a linear or at least monotonic change from the English monolinguals 
through the bilinguals to the Spanish monolinguals would suggest an as- 
similation effect more than a language effect. 

Of the thirty-five non-demographic variables examined, thirteen met the 
first two conditions (spending on social security, spending on foreign aid, 
class self-identification, abortion legal if poor, abortion legal for any rea- 
son, morality of homosexuality, happiness, financial change, party identifi- 
cation, attending religious services, close to Blacks, anti-religious book in 
library, and militarist to teach in college). Of these, two (on homosexuality 
and the anti-religious) met the criteria on only one of the four classifica- 
tions and were dropped from consideration as showing neither a strong nor 
robust pattern. Nine met the criteria on two classifications (social security, 
foreign aid, class, legal abortion for any reason, happiness, party identifica- 
tion, attendance and militarist). Two were statistically significant on all 
four classifications (abortion legal if poor, financial change). Looking at 
both the magnitude of the differences and their pattern across the lan- 
guage-use/ability continuum suggested four items as most likely to indicate 
a language effect (happiness, abortion legal 1f poor, abortion legal for any 
reason, financial change). 

Financial change showed statistically significant variation across all four 
classifications with little difference within the English groups, a large dif- 
ference between bilinguals with Spanish bilinguals indicating much more 
improvement than English bilinguals did and then with Spanish monolin- 
guals indicating much less improvement. Thus, while the bilingual results 
are robust across classification schemes, the difference within the Spanish 
groups indicates that variation is also occurring that is unrelated to ques- 
tionnaire language. 

General happiness showed much lower well-being among the Spanish 
groups than among the English groups, with the largest difference being 
between the English and Spanish bilinguals. The bilingual differences were 
only statistically significant for two classifications, but the other two indi- 
cated a similar pattern. 

The item on whether abortion should be legal for a woman with a very 
low income who cannot afford more children receives much higher support 
among English-language respondents than among those interviewed in 
Spanish, with a large difference between the bilingual groups; this occurs 
for all four classifications. The pattern is similar for being able to obtain le- 
gal abortions for any reason, but the bilingual gap was statistically signifi- 
cant for only two classifications. 

To check whether these detected differences were due to language ef- 
fects, three examinations were carried out. First, linguistically and substan- 
tively similar questions in the GSS were searched for and examined. 
Second, multivariate analyses were carried out on the bilinguals, 
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controlling for assimilation-related variables and other demographics. 
Finally, additional translations were conducted. 

First, in looking for similar GSS items, no suitable items were found for 
financial change (FINALTER). General happiness (HAPPY) had an item 
on marital happiness (HAPMAR) which used both the same key concept 
term (happy/feliz) and the same three response options, differing only in 
applying to married respondents. Marital happiness showed no statistically 
significant variation by language use/ability on any of the four classifica- 
tions, but as with general happiness, marital happiness was rated lower in 
the Spanish versions. 

For abortion there were a total of seven sub-questions and the four not 
selected for initial comparison (abortion if... doesn’t want to marry — 
ABSINGLE, pregnant due to rape — ABRAPE, doesn’t want any more 
children - ABNOMORE, serious defect in the child - ABDEFECT) were 
examined. All seven items showed statistically significant differences by 
language use/ability. Across the bilingual groups, mother’s health showed 
no statistically significant variation on any version, having no more chil- 
dren and birth defect showed marginally significant association on versions 
2 and 4, for any reason and rape showed statistically significant differences 
on versions 2 and 4, and not wanting to marry and being too poor showed 
differences on all four classifications. The pattern thus leans in the direc- 
tion of generally finding lower support among the bilinguals using Spanish 
than among the bilinguals using English, but the strength of the association 
is variable. It tended to be greater among those situations asked about later 
on (ABPOOR, ABRAPE, ABSINGLE and ABANY were 4th-7th) than 
the situations asked about first (ABDEFECT, ABNOMORE and ABHLTH 
were 1st-3rd). This raises the possibility that the meaning shifted across 
items as the introductory phrase receded in memory. However, since the in- 
troductory phrase appeared as an optional re-read on the CAPI screen for 
each follow-up question in both the English and Spanish versions, it is not 
obvious why order would matter. 

Second, multivariate models were run to see if other variables could ex- 
plain the differences in responses between the bilinguals interviewed in 
Spanish and those in English. Various models were tested, but ultimately 
one assimilation variable (born in the United States), one SES variable (ed- 
ucation) and the demographic most closely related to happiness (marital 
status) were used in all models. The models failed to account for the lan- 
guage differences on financial situation or happiness. The differences also 
remain statistically significant for abortions for those with low incomes, 
but the association between language and abortions for any reason was not 
statistically significant with the controls. 

Finally, the items that were identified as showing a statistically signifi- 
cant difference between the bilingual Hispanics that were most likely due 
to language were back translated from Spanish to English by three people, 
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two native Spanish speakers and one native English speaker. Smith then 
compared the back translations to the source English wordings and individ- 
ually discussed the differences with the translators. 

On general happiness the one clear difference between the English and 
the Spanish was in the response categories “very happy, pretty happy, and 
not too happy” and “muy feliz, feliz o no muy feliz”. In Spanish the middle 
option has no modifier. The back translators thought that “feliz” alone 
might be seen as a stronger or more positively leaning category than “pret- 
ty happy” and as such might attract responses away from ‘muy feliz” and 
thus lower the proportion in that category versus ‘very happy”. This would 
be especially true if “pretty” acted in English as a de-intensifier rather than 
an intensifier, but its impact is not clear. There was a consensus that using 
“bastante” as a modifier for the middle option might have been a better 
translation. 

On financial change the main difference was that “financial situation” 
was translated as “situación económica” rather than using the term “finan- 
ciera’. In general, the back translators saw “económica” as appropriate and 
in this question the difference between asking about one’s “economic” ver- 
sus ‘financial’ situation in this context is probably also small in English. 

On the abortion items, the back translations of the seven conditions 
under which a woman might be able to obtain a legal abortion did not sug- 
gest any appreciable difference between the English and Spanish. 
However, the part of the translation of the introductory phrase “Please tell 
me whether or not you think it should be possible for a pregnant woman to 
obtain a legal abortion [...]’ into “Por favor dígame si piensa o no que una 
mujer embarazada debería poder hacerse un aborto en forma legal [...]’ 
was seen as possibly problematic. The Spanish translation for “should be 
possible [...] to obtain’ was “debería poder hacerse”. This was back trans- 
lated in one case as “should be able to have” and as indicating that the 
women should be able to get an abortion for herself, as “ought to be able 
to make happen” and “ought to have”. Thus, the Spanish appears to indicate 
more acceptance of abortion itself rather than just its legal availability. 
This in turn would most likely reduce support for abortion. Moreover, 
while the ‘ought to have” is a mistranslation, it suggests the possibility that 
some respondents may have also misunderstood the item as actually rec- 
ommending that a woman in the situation should obtain an abortion. 

For general happiness and the abortion items, experiments were con- 
ducted on the 2008 GSS with the items being administered to random half 
using their 2006 translations and revised translations that attempted to 
more closely match the original English wordings (Table 10.6).’ For both 
the happiness item and the seven abortion items, the revised Spanish word- 
ings produced statistically different distributions from the original Spanish 
translations and in each case the new translations had distributions closer 
to those exhibited by Hispanics interviewed in English than the 2006 
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Table 10.6 GSS wordings in 2006 and 2008 


Original GSS English 
wording 


2006 Spanish 


2008 Spanish 


Abortion Please tell me Por favor dígame si Por favor dígame si 
(ABDEFECT) whether or not you piensa o no que una piensa que debería 
think it should be mujer embarazada ser posible para una 
possible for a debería poder mujer embarazada 
pregnant woman to hacerse un aborto obtener un aborto 
obtain a legal en forma legal... legal... 
abortion if... 
a) ... there is a a) ... si hay una alta a)... si hay una alta 
strong chance of probabilidad de probabilidad de 
serious defect in the defectos graves en el defectos graves en el 
baby? bebé bebé 
Happiness Taken all together, Tomando todo en Tomando todo en 
(HAPPY) how would you say cuenta, ¿cómo diría cuenta, ¿cómo diría 


things are these 
days — would you 
say you are very 
happy, pretty happy 


que están las cosas 
en estos días? ¿Diría 
que es muy feliz, 
feliz o no muy feliz? 


que están las cosas 
en estos días? ¿Diría 
que es muy feliz, 
bastante feliz o no 


or not too happy? muy feliz? 
a) Very happy a) Muy feliz a) Muy feliz 
b) Pretty happy b) Feliz b) Bastante feliz 


c) Not too happy 


c) No muy feliz 


c) No muy feliz 


Source: General Social Survey (2006-2008) 


translations did. For example, on the 2008 GSS the original Spanish word- 
ing found that 13 per cent were very happy, but the revised Spanish word- 
ing found that 39 per cent were very happy. As expected, adding “bastante” 
to the middle response moved people out of this category and into “muy 
feliz (very happy)’ and this made the options more equivalent to the 
English categories of ‘very’ and “pretty happy”. This indicates that transla- 
tion variation had contributed to the differences observed in 2006. 


Other language comparisons 


Differences in ‘don’t know’ answers 

In addition to the overall differences in distributions examined in Table 
10.5, attention was also focused on don’t know responses (DKs). A scale 
was made of the number of DKs to 25 of the items appearing in Table 
10.5. On average, respondents gave only 0.8 DKs for these 25 items. DK 
levels did not vary by language ability/use on any of the four classification 
schemes. Thus, DK levels do not appear to be related to language, assimi- 
lation or other factors. 
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Table 10.7 Vocabulary score (WORDSUM) by language use/ability scale (mean 
number correct) 


English only English Spanish Spanish English Spanish only Prob. 


First scale 6.50 4.73 6.32 5.10 .000 
Second scale I š 5.74 5.09 .001 
Third scale 6.42 4.53 6.32 5.10 .000 
Fourth scale š; Ú 5.74 5.09 .000 


Source: General Social Survey, 2oo8 
Note: Cells with " indicate that the figures are the same as in the cell above 


Finally, one special battery of questions translated was a ten-item vo- 
cabulary test designed to measure verbal ability (Krosnick & Malhotra 
2007). It was not possible to attempt to develop a Spanish vocabulary test 
that matched the English in reliability and other psychometric properties. 
Instead the ten English target words and each of the five possible responses 
for the word being defined were simply translated into Spanish. Table 10.7 
shows that there are statistically significant differences in vocabulary 
scores across language groups, with the highest score among the English 
monolinguals followed by bilinguals in Spanish, Spanish monolinguals and 
bilinguals in English. This order is surprising since (as discussed below) 
vocabulary has a substantial association with education. Mean years of 
schools completed (13.4) is highest among the English monolinguals, 
which is consistent with their top vocabulary score. But Spanish monolin- 
guals have the lowest education (8.5 years of schooling completed) yet 
score above the English bilinguals with a mean of 13.0 years of schooling. 
Also unexpected was that the Spanish bilinguals outscored the English bi- 
linguals even though they are lower in education (mean 12.0 vs. 13.0 years 
of schooling). This raises the possibility that the vocabulary test is easier in 
Spanish than in English. 

A second comparison of vocabulary scores by language use/ability 
looked at the correlation between vocabulary and education (years of 
schooling completed). For non-Hispanics, Pearson’s r equalled .415, prob. 
=.000. For all Hispanics it was .317 (prob.=.000), for Hispanics inter- 
viewed in English it was .358 (prob.=.000) and for Hispanics interviewed 
in Spanish it was .416 (prob.=.000). Thus, vocabulary has a substantial 
correlation with education for all groups. 


Differences in comprehension and cooperativeness (on the basis of inter- 
viewer assessments) 

Interviewers rated respondents on their comprehension and cooperative- 
ness. As Table 10.4 shows (see appendix at the end of the chapter), the re- 
lationship of language use/ability to these variables is complex. 
Comprehension is rated highest for the English monolinguals and Spanish 
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bilinguals, lower for the English bilinguals, and lowest for the Spanish 
monolinguals. Given that comprehension correlates with education, the 
low ratings for the Spanish monolinguals is not surprising, but the English 
bilinguals show less understanding than would be predicted based on their 
level of education. 

Cooperation is also highest among the English monolinguals and Spanish 
bilinguals and notably lower for both the Spanish monolinguals and 
English bilinguals. Thus, the two bilingual groups are quite different on 
these two interviewer assessments, with Spanish bilinguals rating very high 
on both comprehension and cooperation while English bilinguals are rated 
notably lower on both. One possibility is that English bilinguals may have 
had relatively weak English language skills making interviews with them 
more burdensome. However, the broader and narrower definitions of this 
group show no difference in ratings, which argues against this explanation. 


10.7 Comparability of GSS data across years 


As indicated above, the expansion of the target population to include 
Spanish-speakers notably changed the number and composition of 
Hispanics in the GSS. Consistent measurement over time is a prime goal 
of the GSS and a major challenge (Smith 1988, 2006). For time series from 
the GSS as a whole and for Hispanics in particular to be strictly compara- 
ble, analysis needs to be restricted to the English-language population, 
which means excluding from analysis non-English Hispanics in 2006 and 
subsequent years. 

This can be achieved by using one or both of the language ability meas- 
ures described above: 1) the interviewer’s assessment as to whether the in- 
terview could have been conducted in English and 2) the respondent's 
evaluation of whether they could have done the interview in English. By 
the interviewer measure on the whole sample (versions 1-7), 329 Hispanics 
were interviewed in English and 323 in Spanish. Of the Spanish cases, in- 
terviewers judged that 47 could have been interviewed in English and thus 
276 would be excluded as Spanish monolinguals not eligible under the pre- 
vious English-only criteria. By the respondent measure, 32 say they could 
have done the survey easily in English and 70 with some degree of diffi- 
culty. Thus, at least 222 and a maximum of 292 would be excluded as 
Spanish monolinguals by this measure. 

So far, it is not clear which of these approaches comes closest to dupli- 
cating the coverage of Hispanics using only English-language interviews. 
The best course for now is to try the various alternatives when doing trends 
analysis. Preliminary analysis indicates that the different approaches pro- 
duce similar adjustments. Analysis adding in the 2008 GSS results should 
help to clarify the best adjustment. 
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10.8 Conclusion 


As anticipated, adding Spanish-language interviews notably increased the 
number and proportion of Hispanics in the GSS. In addition, the composi- 
tion of the Hispanic population changed in several notable ways. The add- 
ing of Spanish-language interviews shows that Hispanics are notably less 
assimilated than indicated in the previous English-only samples and also 
differ on several other demographics. This variation across demographics 
is often, but not always, linked to the differences in level of assimilation 
across the language-use/ability groups. The analysis of non-demographics 
further indicates that Hispanics more often than not significantly differ 
across language-use/ability groups. However, no differences in levels of 
opinionation appear across groups. As such, the coverage of the Hispanic 
population is now more complete and its profile more accurate. 

Achieving translation equivalency is one of the most challenging goals 
in cross-lingual research (Smith forthcoming). But as Smith (2008) has 
noted, “Perhaps no aspect of cross-national survey research has been less 
subjected to systematic, empirical investigation than translation.” Even us- 
ing the most rigorous of translation procedures will not always ensure 
translation equivalency (Harkness 2007; Harkness, Pennell & Schoua- 
Glusberg 2004; Smith 2008). To enhance both the process of translation 
and the equivalency of the translation results, quantitative and experimental 
procedures should be utilised. For the 2006 GSS the analysis of the differ- 
ences across language-use/ability groups focusing on the English and 
Spanish bilinguals identified a few items on which language effects may 
be occurring. These suspected translation problems were then tested in 
Spanish-wording experiments in the 2008 GSS. The experiments devel- 
oped new translations, which produced results both different from the orig- 
inal translations and more equivalent to the source English wordings. Such 
experiments are a way of formally testing for translation effects and devel- 
oping better translations. 

Some of the techniques used in developing the Spanish version of the 
2006 and 2008 GSS could be used in multilingual, cross-national studies. 
First, the committee method of translation has some definite advantages 
over the back translation approach, especially since it pays more attention 
to wordings in the target language. Second, for nations with indigenous, 
multilingual populations (e.g. Switzerland, Belgium and Canada) and those 
with appreciable immigrant populations, the procedure of analysing differ- 
ences across monolinguals and bilinguals could be employed to assess the 
equivalence of translation. Finally, experiments using alternative wordings 
could be used to improve translations and to quantify the difference pro- 
duced by alternative translations. 
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Appendix 


Table 10.4 Socio-demographics and interview variables by language use/ability 


Language use/ability 


Socio-demographics English English Spanish Spanish Prob. 
only Spanish English only 
Both parents US (%) 
First 86.0 24.5 11.7 14.1 .000 
Second r J 9.9 15.6 .000 
Third 79.2 20.0 11.7 14.1 .000 
Fourth a Ü 9.9 15.6 .000 
Born US (%) 
First 94.4 57.8 34.3 2.9 .000 
Second š; E 18.5 2.2 .000 
Third 95.4 53.2 34.3 2.9 .000 
Fourth a i 18.5 2.2 .000 
Lived US age 16 (%) 
First 100.0 76.7 54.7 13.9 .000 
Second Ñ ç 38.2 11.1 .000 
Third 99.8 74.4 54.7 13.9 .000 
Fourth úl N 38.2 11.1 .000 
Catholic (%) 
First 43.9 63.5 78.8 78.0 .000 
Second m " 77.8 78.3 .000 
Third 41.7 66.5 78.8 78.0 .000 
Fourth ` $ 77.8 78.3 .000 
Non-entry states (%) 
First 65.0 28.3 1.5 41.6 .000 
Second ü ú 11.9 47.2 .000 
Third 56.3 27.4 15 41.6 .000 
Fourth " " 11.9 47.2 .000 
Large central city (%) 
First 21.0 27.2 41.4 23.1 .262 
Second ji $ 37.0 20.5 .031 
Third 18.4 29.0 41.4 23.1 .092 
Fourth ü i 37.0 20.5 .008 
White (%) 
First 23.2 14.8 15.3 21.8 .002 
Second N y 21.8 20.4 .003 
Third 17.6 15.9 15.3 21.8 .009 
Fourth Ë 3 21.8 20.4 .016 
Male (%) 
First 58.7 53.4 59.2 44.2 .002 
Second y 9 56.5 41.6 .066 
Third 57.9 53.2 58.2 44.2 .009 


Fourth i Š. 56.5 41.6 .016 


234 


Table 10.4 (continued) 


TOM W. SMITH 


Language use/ability 


Socio-demographics English English Spanish Spanish Prob. 
only Spanish English only 
White (%) 
First 23.2 14.8 15.3 21.8 .002 
Second Ñ A 21.8 20.4 .003 
Third 17.6 15.9 15.3 21.8 .009 
Fourth Y š 21.8 20.4 .016 
Male (%) 
First 58.7 53.4 59.2 44.2 .002 
Second s. š 56.5 41.6 .066 
Third 57.9 53.2 58.2 44.2 .009 
Fourth n n 56.5 41.6 .016 
Less than 30 (%) 
First 53.6 34.0 27.6 20.3 .006 
Second É ni 24.3 19.9 .004 
Third 53.3 31.9 27.4 20.3 .003 
Fourth m > 24.3 19.9 .002 
Married (%) 
First 28.8 46.6 58.4 58.1 .010 
Second E " 62.5 56.1 .005 
Third 26.2 49.5 58.4 58.1 .006 
Fourth : Š 62.5 56.1 .003 
No children (%) 
First 46.9 33.4 34.9 15.5 .003 
Second É. Ç. 20.1 17.4 .003 
Third 44.4 32.8 34.9 15.5 .004 
Fourth ` n 20.1 17.4 .003 
Working full-time (%) 
First 43.5 64.6 61.3 52.8 .000 
Second b n 61.7 50.4 .000 
Third 53.2 63.4 61.3 52.8 .000 
Fourth “ Ë 61.7 50.4 .000 
College degree (%) 
First 23.5 16.2 10.9 5.5 .000 
Second a j 8.8 5.1 .000 
Third 22.5 15.8 10.9 5.5 .000 
Fourth E j 8.8 5.1 .000 
Friendly (interviewer) (%) 
First 92.8 76.7 97.1 73.1 .024 
Second R x 92.0 69.2 .002 
Third 86.7 77.1 97.1 73.1 .040 
Fourth Ë b 92.0 69.2 .040 
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Table 10.4 (continued) 


Language use/ability 


Socio-demographics English English Spanish Spanish Prob. 
only Spanish English only 
Comprehension good (interviewer) (%) 
First 94.8 78.6 94.1 59.8 .000 
Second $ N 90.2 52.8 .000 
Third 91.8 77.9 94.1 59.8 .000 
Fourth " ë 90.2 52.8 .000 


Source: General Social Survey, 2oo6 

% Entry states are California, Arizona, New Mexico, Texas, Florida, New York, and New 
Jersey. Non-entry states are the remaining states and Washington, DC 

Note: Cells with “ indicate that the figures are the same as in the cell above 


Table 10.5 Non-demographics by language use/ability 


Language use/ability 


English English Spanish Spanish Prob. 


only Spanish English only 

For death penalty (9) 

(CAPPUN) First 63.3 59.5 45.1 
Second Ñ 5 47.1 
Third 64.8 58.6 45.1 
Fourth i Ë 47.1 

For tough courts (%) 

(COURTS) First 68.7 56.5 70.8 
Second Š É 56.7 
Third 62.0 57.6 70.8 
Fourth Š " 56.7 

For gun regulations (%) 

(GUNLAW) First 66.9 84.3 93.8 
Second E y 91.2 
Third 76.5 82.9 93.8 
Fourth E k 91.2 

More spending for social security (%) 

(NATSOC) First 52.3 65.6 51.2 
Second ‘ Ë 49.1 
Third 52.6 67.0 51.2 
Fourth * K 49.1 

More spending for Parks (96) 

(NATPARKS) First 33.8 43.5 43.8 
Second a" K 40.0 
Third 46.6 39.8 43.8 


Fourth ° " 40.0 
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Language use/ability 


English English Spanish Spanish Prob. 
only Spanish English only 

More spending for foreign aid (9) 

(NATAID) First 6.2 16.0 28.4 25.0 .000 
Second E $ 19.1 28.4 .000 
Third 14.2 14.1 28.4 25.0 .000 
Fourth E y 19.1 28.4 .000 

More spending for science (%) 

(NATSCI) First 41.9 38.7 31.4 29.9 .003 
Second i š 26.4 31.8 .023 
Third 43.0 37.9 31.4 29.9 .003 
Fourth > : 26.4 31.8 .003 

More spending for children (%) 

(NATCHILD) First 50.4 53.5 67.8 50.8 .679 
Second Ñ a 53.8 53.0 739 
Third 49.5 54.2 67.8 50.8 812 
Fourth < ~ 53.8 53.0 .863 

Believing in afterlife (%) 

(POSTLIFE) First 68.2 68.2 80.2 51.7 .002 
Second i : 66.3 50.8 .010 
Third 69.6 67.7 80.2 51.7 .002 
Fourth m Š 66.3 50.8 .012 

Attending church weekly (%) 

(ATTEND) First 29.1 18.6 13.2 16.5 .001 
Second Š š 13.2 17.3 .000 
Third 24.2 19.2 13.2 16.5 .004 
Fourth ü a 13.2 17.3 .000 

Middle/upper class (%) (CLASS) 
First 41.9 33.8 29.3 15.9 .002 
Second ° " 22.5 15.5 .001 
Third 41.4 33.1 29.3 15.9 .001 
Fourth i E 22.5 15.5 .001 

Own home (%) (DWELOWN) 
First 60.0 51.4 35.0 43.7 .067 
Second $ . 45.3 40.7 .073 
Third 51.1 53.8 35.0 43.7 042 
Fourth š Š; 45.3 40.7 .046 

Financially satisfied (%) (SATFIN) 
First 29.0 23.8 22.5 9.3 .000 
Second Í š 19.7 7.2 .000 
Third 24.7 24.8 22.5 9.3 .000 
Fourth " " 19.7 7.2 .000 
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Table 10.5 (continued) 


Language use/ability 


English English Spanish Spanish Prob. 
only Spanish English only 


Better financial position (%) 


(FINALTER) First 41.0 37.3 73.5 48.5 .000 
Second " s 60.8 48.0 .000 
Third 41.6 36.7 73.5 48.5 .000 
Fourth j m 60.8 48.0 .000 

Very satisfied with job (%) 

(SATJOB) First 31.0 53.0 55.9 36.1 .015 
Second F y 50.9 33.2 .002 
Third 38.8 53.0 55.9 36.1 .032 
Fourth ii B 50.9 33.2 .006 

Very happy (%) (HAPPY) 
First 26.6 37.1 18.9 13.7 .000 
Second 7 g 15.0 14.2 .000 
Third 27.8 37.9 18.9 13.7 .000 
Fourth K " 15.0 14.2 .000 

Health excellent (%) (HEALTH) 
First 52.4 27.2 33.6 12.0 .000 
Second " i 26.8 9.3 .000 
Third 44.6 26.8 33.6 12.0 .000 
Fourth Ú; y 26.8 9.3 .000 

Fear of walking at night (%) (FEAR) 
First 22.3 37.0 55.4 50.0 .082 
Second " ü 46.8 52.0 .077 
Third 22.4 39.0 55.4 50.0 .042 
Fourth i * 46.8 50.0 .039 

Great deal of confidence in 

federal government First 20.5 15.4 19.3 13.9 .351 

(CONFED) Second " " 10.9 16.8 .018 
Third 14.6 17.0 19.3 13.9 .025 
Fourth " i 10.9 16.8 .001 

Great deal of confidence in 

Congress (%) (CONLEGIS) First 26.9 11.7 27.1 20.0 .066 
Second " z 23.5 19.8 .157 
Third 19.1 13.0 27.1 20.0 .093 
Fourth y ” 23.5 19.8 213 

Great deal of confidence in 

religion (96) (CONCLERG) First 11.1 24.2 19.3 329 447 
Second E 26.5 33.1 .510 
Third 12.4 25.2 19.3 32.9 .060 


Fourth y š 26.5 33.1 .073 
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Language use/ability 


English English Spanish Spanish Prob. 
only Spanish English only 

Homosexuality always wrong 

(96) (HOMOSEX) First 44.9 51.0 73.9 68.4 000 
Second s 55.6 74.1 .000 
Third 48.0 50.6 73.9 68.4 .000 
Fourth š " 55.6 74.1 .000 

Premarital sex always wrong (%) 

(PREMARSX) First 21.0 13.4 11.4 26.5 .003 
Second ; Š 20.6 38.2 .001 
Third 22.4 12.1 11.4 26.5 .002 
Fourth * i 20.6 38.2 .002 

Disagree women not suited for 

politics (%) (FEPOL) First 87.1 74.9 66.0 54.2 .007 
Second Ü Ú 60.4 54.4 .013 
Third 78.6 76.9 66.0 54.2 .008 
Fourth Š ~ 60.4 54.4 .015 

For abortion if woman has low 

income (%) (ABPOOR) First 50.0 50.1 6.6 20.9 .000 
Second 3 " 15.1 21.2 .000 
Third 54.3 48.4 6.6 20.9 .000 
Fourth i Š; 15.1 21.1 .000 

For abortion for any reason (%) 

(ABANY) First 50.0 42.7 21.2 1537, .000 
Second ñ š 19.6 15.0 .000 
Third 46.5 43.1 21.2 15.7 .000 
Fourth A " 19.6 15.0 .000 

For abortion if women’s health threatened 

(%) (ABHLTH) First 72.7 86.2 93.3 72.9 .000 
Second - š 80.8 72.5 .000 
Third 78.4 85.9 93.3 72.9 .009 
Fourth i : 80.8 72.5 .007 

Minorities should work way up (%) 

(WRKWAYUP) First 65.8 63.0 84.0 85.5 .004 
Second E i 87.2 84.3 .003 
Third 62.2 64.1 84.0 85.5 016 
Fourth Ú. š 87.2 84.3 .012 

Close to blacks (8+9) (%) 

(CLOSEBLK) First 38.8 16.6 27.8 4.5 .000 
Second j Ú 14.0 3.9 .000 
Third 38.5 13.7 27.8 4.5 .000 
Fourth E š 14.0 3.9 .000 
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Table 10.5 (continued) 


Language use/ability 


English English Spanish Spanish Prob. 
only Spanish English only 

Close to whites (%) (8+9) 

(CLOSEWHT) First 48.1 28.6 15.4 12.8 .001 
Second " $ 17.0 11.5 .000 
Third 38.8 29.6 15.4 12.8 .010 
Fourth " pe 17.0 11.5 .001 

Allow anti-religionist book (%) 

(LIBATH) First 84.5 60.7 84.6 60.2 137 
Second Ü " 78.6 56.4 .010 
Third 79.4 59.4 84.6 60.2 .124 
Fourth š P 78.6 56.4 .009 

Allow militarist to teach college (%) 

(COLMIL) First 62.2 48.9 33.6 30.4 .002 
Second i š 25.7 32.7 .002 
Third 59.2 48.3 33.6 30.4 .004 
Fourth " 2 25.7 32.7 .004 

Voted for president in 2000 (%) 

(VOTE00) First 46.8 38.1 30.6 12.1 .000 
Second y F 25.0 9.9 .000 
Third 48.1 36.6 30.6 12.1 .000 
Fourth : t 25.0 9.9 .000 

Voted for president in 2004 (%) 

(VOTE04) First 77.9 50.5 45.3 11.4 .000 
Second É: 5 36.0 6.9 .000 
Third 75.1 48.3 45.3 11.4 .000 
Fourth Š >) 36.0 6.9 .000 

Democratic (9) 

(PARTYID) First 25.1 43.6 54.0 22.1 .000 
Second A Ë 40.0 20.4 .000 
Third 33.0 42.7 54.0 22.1 .000 
Fourth i š 40.0 20.4 .000 


Source: General Social Survey, 2oo6 


Note: Cells with " indicate that the figures are the same as in the cell above 
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Notes 


1 The GSS was sub-divided into seven random sub-samples. This number and all others in 
this report are weighted figures (Davis, Smith & Marsden 2009). 

2 On version 7 there were an additional 225 Hispanic respondents, 109 were interviewed in 
English and 116 in Spanish. Because those interviewed in English on version 7 were not 
asked about Spanish-language ability, these cases are not used in most of the subsequent 
analysis. 

3 Spanish respondents refer to interviewees that carried out the interview in Spanish. 

4 ETHNIC is coded ‘From what countries or part of the world did your ancestors come? IF 
MORE THAN ONE COUNTRY NAMED: Which one of these countries do you feel 
closer to? ETH1, ETH2 and ETH3 are the up to three ethnic origins mentioned. 
HISPANIC asked ‘Are you Spanish, Hispanic or Latino/Latina?” 

5 The four language use/ability classifications generally showed the same patterns and only 
in the exceptional case where they made a difference will their specific results be referred 
to. 

6 Seven states with large Hispanic populations which tend to be points of entry into the 
United States were classified as ‘entry states’. They are Arizona, California, Florida, New 
Jersey, New Mexico, New York and Texas. The remaining states and the District of 
Columbia are ‘non-entry states’. 

7 For another rare example of surveys using experiments involving translations see Shrout, 
Alegria, Canino, Guarnaccia, Vega, Duan and Cao (2008). 
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11 Under-representation of foreign minorities in 
cross-sectional and longitudinal surveys in 
Switzerland 


Oliver Lipps, Francesco Lagana, Alexandre Pollien 
and Lavinia Gianettoni 


11.1 Introduction 


Most Western societies have seen massive international migration inflows 
during the past decades. Unlike the typical migrant of the 1960s, however, 
today’s migrants can hardly be standardised in terms of skills, gender and 
country of origin. This results in a high heterogeneity in terms of educa- 
tional levels, social positions and gender (Koser & Lutz 1998; Kofman, 
Phizaclea, Raghuram & Sales 2000). 

Many authors (see Wimmer & Schiller 2002; Chernillo 2006; Wimmer 
& Min 2006) have addressed the issue of whether social surveys are (still) 
able to represent foreigners both with respect to heterogeneity of their cul- 
tural background, educational level and position in the hosting country’s 
social structure. There is evidence that foreign minorities in general (espe- 
cially the more marginal and vulnerable individuals) are under-represented 
among respondents (Vandecasteele & Debels 2007). There are several rea- 
sons for this: under-coverage in sampling frames (Lipps & Kissau 2012), 
higher non-response rates in cross-sectional surveys (Deding, Fridberg & 
Jakobsen 2008; Feskens, Hox, Lensvelt-Mulders & Schmeets 2006, 2007; 
Camarota & Capizzano 2004; Eisner & Ribeau 2007; Jakobsen 2004; 
Nielsen & Pedersen 2000) and higher attrition rates in longitudinal surveys 
(Lipps 2007; Peracchi & Depalo 2006). To analyse under-representation, 
the literature has either focused on selected minorities (Deding et al. 2008) 
or used a rough distinction between Western and non-Western minorities 
(Feskens et al. 2007) that is not likely to adequately capture the increasing 
heterogeneity. 

In this chapter we would like to shed light on the question of whether 
there is a bias in Swiss social surveys regarding the representation of for- 
eign minorities. If so, are there sub-categories according to social class or 
education within different nationality groups that make them especially 
prone to being under-represented? A related question refers to the extent to 
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which this possible bias changes depending on the type of survey: does the 
under-representation found in cross-section surveys increase in panel sur- 
veys? We use data from three Swiss social surveys: the Swiss Household 
Panel (SHP), the Swiss National Labour Force Survey (LFS) and the 
Swiss sample of the European Social Survey (ESS). We use three different 
surveys to check if possible patterns of under-represented national minor- 
ities are independent of survey design features. In addition, using panel da- 
ta like the SHP and the rotating LFS panel allows analysing attrition as a 
second dimension of under-representation. 

According to the 2008 Foreign Population Structure and Migration 
Statistics (PETRA),' the foreign population in Switzerland amounted to 
1.67 million, compared with a total population of 7.70 million. The main 
minorities are from former Yugoslavia,? representing 19 per cent of the to- 
tal foreign population, Italians (17 per cent), Germans (14 per cent) and 
Portuguese (11 per cent). 

There are three reasons why Switzerland is an ideal case to analyse the 
degree to which the heterogeneity of migrant populations is represented in 
surveys. In the first place, Switzerland has a long history as a country of 
immigration. According to the 2008 Swiss Labour Force Survey about 31 
per cent of the total population aged 15 or older has a migration back- 
ground. Since the end of the Second World War a stable political situation, 
prosperity, an undamaged industrial sector and the shortage of labour force 
were all important pull factors that attracted low-skilled workers from other 
West European countries. In 1970, the number of foreigners in Switzerland 
amounted to over one million (Piguet 2004). More than half of them were 
Italians, followed by Germans and Spanish. After a reduction in the inflow 
of immigration following the 1973 economic crisis, the period between 
1985 and 1992 was again characterised by rising immigration flows from 
Portugal and former Yugoslavia. As a result of these two different waves 
of unskilled immigration, there is a great diversity in the immigrant popu- 
lation with respect to the countries of origin. 

In the second place, the reasons behind migration are an important factor 
that characterise the immigrant population living in Switzerland. While in 
the 1950-1960s migrations were mostly labour-driven, there was a strong 
increase of migrations due to non-economic factors, such as family reunifi- 
cations after 1980. The last important factor that produced a high heteroge- 
neity in Switzerland is the presence of highly skilled migrants, in part as a 
consequence of the concentration of the headquarters of many international 
firms within Swiss borders (Gross 2006) and also because of the shortage 
of a highly educated autochthonous labour force caused by a selective edu- 
cational system. 

This chapter is organised as follows. In the first section we describe the 
theoretical framework used to analyse the representation of foreign minor- 
ities both in cross-section and longitudinal surveys. Next, we formulate 
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hypotheses to find out whether educational levels and social class help to 
explain the under-representation of foreigners in surveys. We then intro- 
duce the data and the models used to analyse under-representation in Swiss 
cross-sectional and panel surveys, after which we graph and present the 
under-representation of certain foreigner groups. In the final section we use 
multivariate models to analyse under-representation, interpret the results 
and draw conclusions on the implications of these results for survey design 
and for the analysis of survey data. 


11.2 Determinants of under-representation 


To better understand under-representation of national minorities we adopt 
the ‘Total Survey Error’ approach (Groves 1989), according to which 
under-representation mostly stems from both under-coverage problems of 
the sampling frame and from non-response (Groves 1987). 

Under-coverage arises “if the population from which the sample of cases 
is drawn is incomplete” (Corbetta 2003: 79) such that parts of the target 
population are not adequately represented. For example, with regard to tel- 
ephone surveys, national minorities usually have a lower likelihood of 
owning a land-line telephone and — even if they do — being listed in tele- 
phone books (Lipps & Kissau 2012). With respect to the accuracy of sam- 
pling frames, there is evidence that more instability with respect to living 
arrangements and residential situations is a likely cause of under-represen- 
tation of foreign minorities (Centraal Bureau voor de Statistiek 2004; 
Camarota & Capizzano 2004). 

Non-response is either due to non-contact or non-cooperation once con- 
tacted. In panel studies, this includes not only (initial) non-response in the 
first wave, but also attrition, which means temporary or definitive drop-out 
of eligible sample members who once participated in the survey. Attrition 
is a threat to the long-term survival of a panel survey for two reasons: first, 
the sample size may become too small for detailed analyses and, second, 
attrition tends to leave an increasingly selective sample (Lipps 2007). It is 
mainly this second reason why non-response in cross-sectional surveys 
may be different from non-response in panel surveys after a number of 
waves of participation. 

Higher levels of non-contact among foreigners/immigrants are often due 
to more non-standard work times (Deding et al. 2008; Feskens et al. 2006, 
2007). To be able to cooperate, survey language competence is of course 
an obvious requirement.? At the least, poor language competences add con- 
siderably to the survey burden and may discourage cooperation of non-na- 
tive speakers. As for more survey-related reasons, Groves, Fowler, Couper, 
Lepkowski, Singer and Tourangeau (2004) identify topic interest as one 
important issue: one reason for non-participation is probably that issues 


244 OLIVER LIPPS, FRANCESCO LAGANÀ, ALEXANDRE POLLIEN AND LAVINIA GIANETTONI 


relevant to the host country may not concern the minority interests. In ad- 
dition, according to the “social exclusion” theory, the more socially ex- 
cluded tend to be under-represented in surveys (Groves & Couper 1998; 
Groves et al. 2004; Stoop 2005). Both foreigners and socially vulnerable 
people may have higher non-response rates, possibly because of more em- 
barrassment about sensitive topics or because they do not like to disclose 
societal exclusion issues. Additionally, foreign respondents might not feel 
concerned enough to participate. 

As for socio-economic correlates, Schmeet and Michiels (2003) stress 
the importance of economic status to explain national minorities’ higher 
non-response rates. Feskens et al. (2006) identify variables like the degree 
of urbanisation, employment, socio-economic status and educational level. 
Jakobsen (2004) and Nielsen and Pedersen (2000) explain higher non-re- 
sponse rates among minorities in the Netherlands by educational level and 
employment status. Feskens et al. (2007) show that the effect of nationality 
is almost entirely mediated by the degree of urbanisation, since the latter is 
likely to influence non-contact rates. Deding et al. (2008) find that in 
Denmark the different non-response rates between immigrants and natives 
persist even after controlling for socio-economic variables. 

These different aspects show that the reasons for under-representation in 
surveys may be different for foreigners from different origins and cultures. 
The chapter analyses the extent to which educational levels and (occupa- 
tional) social class are related to under-representation. The first hypothesis 
is that under-representation is completely independent of social class and 
education. The second hypothesis is that under-representation of foreign 
minorities observed in Swiss surveys is completely mediated by education 
and/or social class, that is, the more marginal condition of foreign minor- 
ities would explain their under-representation. The third, alternative, hy- 
pothesis is that socio-economic factors such as education and social class 
moderate the under-representation of foreign minorities. Thus, under-repre- 
sentation would be higher for minorities with less education and those in 
disadvantaged social classes. 


11.3 Data, methods and variables 
Data 


We use data from three Swiss social surveys: the Swiss Household Panel 
(SHP), the Swiss National Labour Force Survey (LFS), and the Swiss sam- 
ple of the European Social Survey (ESS).* The SHP is a yearly multi-topic 
CATI household panel survey about living conditions, attitudes and social 
change, conducted by the Swiss Centre of Expertise in the Social Sciences 
(FORS). The SHP is representative of the residential population living in 
private households, irrespective of duration of stay, permit status and 
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nationality. All household members 14 years of age and older are eligible 
to be interviewed. The first sample was taken in 1999 with 5.074 com- 
pleted households and a refresher sample was added in 2004 with 2,578 
completed households, both sampled from the telephone register. Within 
each household, one member is assigned to be the reference person as a 
source of information on household-related characteristics. Survey lan- 
guages offered comprise the three Swiss national languages (German, 
French and Italian). 

The LFS is a yearly individual survey on labour market participation, 
conducted by the Swiss Federal Statistical Office (SFSO). The LFS is rep- 
resentative of the permanent resident population aged 15 years and older, 
including foreigners in Switzerland with a stay permit of at least one year. 
Each individual is followed during five years (rotating panel component). 
The survey started in 1991 with a sample of 16,000 individuals, sampled 
from the telephone register. Modifications during the past years added a 
sample of about 15,000 drawn from the central register of foreigners to the 
standard LFS sample in 2003. In the same year, questionnaires in Serbo- 
Croat, English and Albanian were introduced, with additional question- 
naires in Turkish and Portuguese from 2005 on. The interview involves 
CATI if the language chosen is one of the three national languages, while 
it involves CAPI if another language is chosen. 

The ESS is an individual biennial face-to-face cross-sectional survey on 
changing public attitudes and values in Europe, with the Swiss part con- 
ducted by the Swiss Centre of Expertise in the Social Sciences (FORS). 
The ESS started in 2002 and is representative of the residential population 
older than 14 years of age, irrespective of duration of stay, permit status 
and nationality. The effective sample size is to 1,500 individuals for 
Switzerland. While the 2002, 2004 and 2006 Swiss samples were drawn 
from the telephone book, for the 2008 round the register of postal ad- 
dresses was used. Because the sample is household-based, the target person 
is randomly selected within sampled households. This procedure also re- 
quires consent of the contact person of the household. Survey languages 
offered comprise the three national languages. 


Methods 


To answer our research questions, we distinguish between cross-sectional 
and longitudinal dimensions. Analysing the composition of final respond- 
ents and how different subgroups of the population are represented in 
cross-sectional surveys requires background information such as that in- 
cluded in administrative registers or information about the gross sample. 
Since this information is not available in the surveys considered, the only 
way to deal with this problem is to compare survey respondent statistics 
with statistics from other data sources, such as the census. The analysis of 
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how attrition affects the representativeness of panel survey data, however, 
can benefit from the information collected in earlier waves of the panel. 

While data from the Swiss census stem from 2000, for the three surveys 
different reference years are used. The reason for this choice is that the sur- 
veys have different designs, and substantial modifications were made in 
the ESS and particularly in the LFS. Two criteria determine the selection 
of the survey reference year. First, we use the years in which the design of 
the three surveys was as similar as possible, in order to avoid patterns of 
under-representation due to the particular design of a survey. Second, we 
use the years which are as close to 2000 as possible in order to minimise 
time effects. Following these two criteria, the reference year for the cross- 
sectional analysis of the LFS was 2002. The reason for this, as mentioned 
in the previous section of the chapter, is that in 2003 the LFS added a sub- 
sample of foreigners and new languages for conducting interviews. Due to 
the small sample size of each individual sample of the Swiss part of the 
ESS, the analysis is based on the pooled data from the first four rounds of 
the surveys (2002-2008) in order to have a sufficiently large number of 
cases. 

To analyse cross-sectional representation, we use the ratio between the 
odds of a randomly selected individual being represented in the survey 
sample and the relative odds in the population (census value) as a measure 
of bias. The advantage of taking odds instead of percentages is that odds 
are independent of the marginal distribution of the observed variables. For 
example, if we consider education by nationality, the odds allow estimating 
the magnitude of the bias of the two variables in the population independ- 
ently from the marginal distribution of education and nationality. Odds 
close to 1 indicate that the representation of a given category is unbiased, 
odds smaller than 1 indicate under-representation and odds greater than 1 
show over-representation. 

For the longitudinal analysis of the LFS, the year 2001 was chosen as 
the starting year. For the SHP the reference year was the first year of the 
survey (1999), because in later years attrition might have affected the sam- 
ple more strongly than the (small) one-year time effect. To analyse attrition 
bias, we use logistic regression models. 


Variables used to analyse the patterns of under-representation 


We compare the representation of different population groups in the three 
surveys with that in the population using three variables that are included 
in the census 2000: nationality, education and social class. Because of dif- 
ferent survey sample sizes, these variables need to be grouped in different 
ways. With respect to the foreign minority groups, the LFS data allows dis- 
tinction between Swiss, Italians, French, Portuguese, Germans, Turkish, 
former-Yugoslavians and Albanians, those from the rest of Europe, 
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Africans, Americans, Asians and Oceanians. In the other surveys we distin- 
guish between the Swiss nationals, nationals from neighbouring countries 
(Germans, Austrians, Liechtensteiners, French and Italians) and nationals 
from a non-neighbouring foreign country. 

With respect to educational levels, we distinguish between “no educa- 
tion/secondary I’, ‘secondary II’ and ‘tertiary’. Also, we use a fourth cate- 
gory for respondents who are still in training. For social class, we use the 
ISCO (International Standard Classification of Occupations), recoded into 
three categories: upper non-manual (ISCO codes 1, 2 and 3), lower non- 
manual (ISCO codes 4 and 5) and manual (ISCO codes 6, 7, 8 and 9). 
Individuals who are not in the active labour force are gathered in a fourth 
category. We quantify social class by occupational class because from a 
theoretical point of view occupation is the main determinant of individual 
social class position (Crompton 2008). Methodologically, we used the 
ISCO code because this variable is available in the census and in the three 
datasets we analysed. 

In the longitudinal analysis, the availability of data from previous waves 
of the SHP and the LFS (rotating panel sample) allows incorporating addi- 
tional variables in the analysis such as the length of stay in Switzerland. 
These additional variables will be explained in the sections where the dif- 
ferent models to analyse under-representation are displayed. 


11.4 Education, social class and the representation of foreign 
minorities 


In this section we analyse whether there is evidence of under-representa- 
tion of national minorities in the selected Swiss surveys and if so, the ex- 
tent to which these are linked to variables such as social class and educa- 
tional levels. 

Figure 11.1 displays the odds ratios for national minorities in the three 
surveys compared to the census 2000. All three surveys show a similar pat- 
tern of under-representation of foreigners, especially of those from coun- 
tries that are not geographically close to Switzerland. 

In addition, the availability of more disaggregated data for the LSF let 
us see that the most under-represented groups are Turkish, former- 
Yugoslavians and Albanians, Africans and Asians. In all three surveys, mi- 
norities that are best represented are those from the neighbouring countries 
Italy, France and Germany. 

The next step is to examine whether the bias against some foreign 
groups is constant within nationality or whether differences depending on 
educational level or social class make some particularly prone to under-rep- 
resentation. Analysing the interaction between the socio-economic varia- 
bles, education, social class and national minorities’ under-representation 
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National minorities' representation (odds ratios) in the Swiss 


Figure 11.1 
Household Panel 1999, the European Social Survey 2oo2-2oo8, and the 
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helps to explain the nature of the bias and the interplay between different 
dimensions. 

Figures 11.2-11.4 show the odds ratios of the representation of sub- 
groups of Swiss nationals and of different foreign minority groups accord- 
ing to education and social class in the LFS 2002, the ESS 2002-2008 and 
the SHP 1999, relative to the 2000 census. As mentioned above, values be- 
low 1 indicate under-representation and values above 1 show over-repre- 
sentation. For example, if we consider the representation of the Portuguese 
in the LFS by education (Figure 11.2, upper graph), we find an over-repre- 
sentation of individuals with tertiary education, an almost correct 


Figure 11.2 National minorities’ representation (odds ratios) by education and 
social class in the Swiss Labour Force Survey 2002 
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representation of the population with secondary II education and an under- 
representation of individuals with no education or secondary L 

The data of the LFS indicate a general under-representation of those na- 
tional minorities that constitute the backbone of the more recent migration 
(former-Yugoslavs and Turkish), segments of Spanish and Portuguese (with 
a lower education level) and those more distant from the core of Swiss so- 
ciety (non-Europeans). 

With respect to education, all three surveys show that the under-repre- 
sentation is more prevalent for individuals with low education, especially 
for those from non-neighbouring countries. For example, 80 per cent of 
people from non-neighbouring countries with low education are missing in 
the ESS final set of respondents (Figure 11.3, lower graph) and 60 per cent 
of them are not represented in the SHP (Figure 11.4, upper graph). The ed- 
ucational bias is especially strong in the ESS (but also in the SHP) and also 
affects the Swiss population. As for the differences between surveys, we 


Figure 11.3 National minorities’ representation (odds ratios) by education and 
social class in the European Social Survey 2002-2008 


ESS - Social class 


Odds ratios 


Swiss Neighbouring country Other country 
-9- Manual Lower non-manual 
t= Higher non-manual © Not active or unemployed 


ESS - Education 


1,5 

gi 
8 
Nn 
mej 

60,5 

0 

Swiss Neighbouring country Other country 
= Not completed / Secondary | Secondary l| =te=Tertiary 


Source: ESS 2002-2008 (Swiss data) 


UNDER-REPRESENTATION OF FOREIGN MINORITIES 251 


Figure 11.4 
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are not able to find systematic survey characteristics like difficulty, length, 
topic, incentives or sponsorship that are able to explain this result. 

The picture is quite similar when we consider the LFS (Figure 11.2, 
upper graph). The graphs show that there is no systematic under-represen- 
tation of foreigners with a mid- or high-level education. 

With respect to social class, non-manual workers are generally better 
represented than manual workers in the ESS (Figure 11.3, upper graph) 
and the SHP (Figure 11.4, lower graph). The LFS (Figure 11.2, lower 
graph) suggests that foreigners from at least Turkey, the former Yugoslavia 
and Asia are under-represented, independent of social class. 

Summarising, in the three surveys, poorly educated foreigners tend to be 
under-represented. This pattern is more pronounced with nationals from 
‘distant’ countries and less so among foreigners coming from Swiss 
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neighbour countries. The extent to which education is related to representa- 
tion varies across national groups. For example, highly educated 
Portuguese are over-represented, whereas Turkish individuals of all educa- 
tion levels are under-represented in all three surveys. Unlike education, so- 
cial class does not seem to have a strong moderation effect. The implica- 
tion of this is that foreign minorities tend to be under-represented irrespec- 
tive of their social class. In the next section we analyse whether these 
tendencies cumulate or level off when doing an attrition analysis. 


11.5 Analysis of representation of foreign minorities in 
longitudinal surveys 


Representation of foreign minorities in the SHP 


In the current section we analyse if, after the first wave of a panel survey, 
under-representation of national minorities increases due to attrition. In ad- 
dition to social class and education, we include other social variables 
known to correlate with attrition. The time foreigners have lived in 
Switzerland, as a proxy for “acculturation”, is controlled for.° Conceptually 
we follow Voorpostel and Lipps (2011), who also used data from the SHP. 
Like them, we model obtaining contact based on the household level (i.e. 
obtaining contact with the household reference person) and obtaining coop- 
eration on the individual level (i.e. obtaining cooperation with each house- 
hold member eligible for interview). As is common in surveys, we model 
obtaining cooperation with household members conditional on having ob- 
tained contact. Unlike in the research of Voorpostel and Lipps (2011), how- 
ever, we also keep interview-eligible households and individuals who only 
participated during one wave.’ 

The attrition behaviour of households and individuals can be classified 
in the following patterns. Households (individuals) that were successfully 
contacted (completed the individual questionnaire) in every wave are coded 
‘always in’. Households (individuals) not successfully contacted (did not 
complete the individual questionnaire) in all waves, but which completed 
the individual questionnaire in at least the last two, are coded as ‘tempo- 
rary out’. ‘Dropped out’ are households (individuals) that were not con- 
tacted (did not respond) in the last two waves. 

Table 11.1 shows the outcome of fieldwork regarding establishing contact 
and obtaining cooperation shown by the different attrition groups. It also 
shows the pattern according to nationality, using the grouping 1) Swiss na- 
tionals, 2) nationals from neighbouring countries and 3) other nationalities. 

The respective first values listed are the frequencies. For example, we 
have 6,101 households among all 6,894 Swiss households that always re- 
port. The respective second value is the expected frequency if the response 
category variable were independent of the nationality variable. Taking the 
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population category from the previous example, we would expect 6,020 
Swiss households to be in the upper left cell if the nationality was inde- 
pendent of the response category. This figure is simply the product of the 
respective marginal sums, divided by the total sum (here 6,843 times 6,894 
divided by 7,837). The respective third figure is the standardised difference 
between the actual and the expected (under the independency hypothesis) 
figures. This is the contribution of the cell to the chi squared. This latter 1s 
a measure of the discrepancy between the expected frequency (assuming 
independency of the crossed variables) and the actual frequency. All chi 
squared cells add up to the total chi squared (lower right figure). 

The total chi squared value on the household level (102.6) is significant 
on the 1 per cent confidence level; the total chi squared value on the indi- 
vidual level (11.2) on the 5 per cent confidence level. The large difference 
of the two chi squared values gives a first indication that nationality has a 
greater influence on differences in contact rates than in cooperation. As far 
as response patterns with respect to household contact rates are concerned, 
by far the highest chi squared contribution comes from dropped-out for- 
eigners other than from a neighbouring country (67.3). This group also 
exhibits the highest likelihood of being temporary out. Also, foreigners 
from neighbouring countries have slightly higher drop out and temporary 
out rates. On the individual (cooperation) level, we find that the highest 
chi squared contribution stems from foreigners from neighbouring coun- 
tries who drop out temporarily (6.5). Foreigners from non-neighbouring 
countries have the highest drop out rates. 


Table 11.1 Household and individual response patterns by nationality 


Household (Contact) Individual (Cooperation) 


Swiss Neighbour Other Total Swiss Neighbour Other Total 


Always in 6,101 420 322 6843 7,788 478 420 8,686 
6,020 443 381 7,753 505 428 

1.1 1.2 9.1 11.3 0.2 1.5 0.1 1.8 

Dropped out 417 48 75 540 453 32 33 518 
475 35 30 462 30 26 

7.1 4.9 67.3 79.3 0.2 0.1 2.2 2.5 

Temporary out 376 39 39 454 1,490 124 84 1,698 
399 29 25 1,516 99 84 

1.4 3.2 7.5 12.0 0.4 6.5 0.0 6.9 

Total 6,894 507 436 7,837 9,731 634 537 10,902 

9.6 9.2 83.8 102.6 0.8 8.0 2.3 11.2 


Source: SHP 2000-2009 

Note: First values in crossed cells are the frequencies, the second the expected frequency if 
the response category variable were independent of the nationality variable, and the third 
the cell chi squared contribution to the total chi squared. 
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We conduct multivariate analysis to check if the lower household con- 
tactability of foreigners from non-neighbouring countries due to dropping 
out permanently and the lower individual cooperation of foreigners from 
neighbouring countries, due to dropping out temporarily, persist if addi- 
tional variables are controlled for. The following independent variables are 
controlled for in the models, though for the sake of simplicity, the coeffi- 
cients are not listed in Tables 11.2 and 11.3: 

— whether the household could not be contacted in at least one wave 
(comparing them to ‘always contacted’ households [only individual 
level])? 

— sample to which the household corresponds (1999 versus 2004) 

— number of waves in the panel 

— whether the person is household reference person or not, as defined in 
section 11.2 

— whether there is a partner in the household 

— whether there are any children under 18 years of age in the household 

— age group 

— gender 

— ownership of a house/flat (using tenants as the base category) 


According to the high cell chi squared (Table 11.1), for the household con- 
tact models we compare dropped out only with all other households; like- 
wise, for the individual cooperation models temporary out with all other in- 
dividuals. We use logit models for the different attrition groups (‘temporary 
out’ vs. ‘always in’ and ‘drop out’ vs. ‘always in’), distinguished by the 
set of independent variables used. 

Model 1 includes control variables already described plus nationality, 
education and whether the person is employed. Nationality classifies indi- 
viduals in the three groups already mentioned: Swiss nationals (base cate- 
gory), nationals from neighbouring countries and remaining nationalities. 
Unlike in the cross-sectional analysis, cases are classified according to two 
educational levels: above and below high school level, the latter being the 
reference category. 

The model also includes an interaction term of nationality with the num- 
ber of years living in Switzerland classified in four groups, 0-2 years (base 
category), 3-5, 6-10 and more than 11 years. Length of stay does not only 
measure residence duration, but also allows capture of the different cohorts 
of migrants. For instance, people who migrated from former Yugoslavia in 
the 1990s (measured as residing in Switzerland for fourteen years in 2004) 
are different than those who did so in 2000 (measured as residing in 
Switzerland for four years in 2004). 

Table 11.2 shows the results of the logit models that compare dropped- 
out households with households that were always in, regarding establish- 
ing contact. On the household level, similar to the bi-variate cross- 
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tabulated results, we find more dropping out due to non-contact among 
households with a nationality from a non-contiguous country, compared 
to neighbouring ones and especially the Swiss, also in the controlled logit 
model. The acculturation factors do not seem to have an effect on the 
contactability of other foreign groups. However, as for neighbours, a lon- 
ger stay in Switzerland appears to increase contactability: although the ac- 
culturation terms are not significant. While a short stay (under five years) 
makes the nationality main effect term significantly positive, a longer ac- 
culturation leads to better contactability. Additionally, in line with find- 
ings from Voorpostel and Lipps (2011), higher education increases the 
contactability of households. 

Next, we model cooperation at the individual level, where we control 
for clustering within households by using a random intercept modelling ap- 
proach. In this case, based on the bi-variate analysis done in the previous 
section, the comparison is between ‘temporary out’ versus “always in’ 
households. 

While model 1b includes the same control and research variables as the 
ones already explained for model la, in addition to the ones included in 
model 2a, model 2b incorporates two additional independent variables re- 
lated to social and political participation. The first of these is whether the 
individual participates in clubs or associations, whereas the second is the 
interest in politics, measured on a 0-10 scale. Table 11.3 shows the results 
of logit models 1b and 2b. 


Table 11.2 Coefficients of logit model: ‘drop-out’ versus “always in’ households 


‘Drop out’ versus ‘always in’: Contact Model 1a Model 2a 
Nationality: neighbour 0.39 0.91** 
Nationality: other 0.79%* 0.98*%* 
Education: high school? -0.38* -0.38* 
Education: above high school -0.58%* -0.60** 
Employed 0.14 0.13 
Neighbour x 3-5years 0.00 
Neighbour x 6-10 years -0.70 
Neighbour x 11+ years -0.96 
Other x 3-5years -1.10 
Other x 6-10 years 0.30 
Other x 11+ years -0.40 
Log Likelihood -908 -904 
McFadden Pseudo R? 0.145 0.149 


Source: SHP 2000-2009 (N=7,078) 

* Reference category: below high school level. Interaction terms between nationality and 
education levels have no effect 

Note: Models controlled for sample (1999 vs. 2004), number of waves, partner in household, 
children in household, gender, age-group of reference person, owner vs. tenant of the house. 
Significance levels: * p < 0.05, ** p<0.01 
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Table 11.3 Coefficients of 2-level logit model: ‘temporary out’ versus ‘always in’ 


individuals 

“Temporary out' versus “always in”: Cooperation Model 1b Model 2b 
Nationality: neighbour 0,45** -0.46 
Nationality: other 0.06 -0.34 
Education: high school -0.03 0.00 
Education: above high school -0.09 -0.02 
Employed 0.21% 0.20* 
Neighbour x 3-5 years 1.48** 
Neighbour x 6-10 years 0.91 
Neighbour x 11+ years 0.92* 
Other x 3-5 years 0.74 
Other x 6-10 years 0.32 
Other x 11+ years 0.40 
Participation in clubs/associations -0.11 
Political interest (0-10 scale) -0.02 
Log Likelihood -4,292 -4,283 
Rho 0.30 0.30 


Source: SHP 2000-2009 (N=10,384) 

Note: Models conditional on contact success of household. Models controlled for sample 
(1999 vs. 2004), number of waves, partner in household, children in household, gender, 
age-group, owner vs. tenant of the house. Significance levels: * p < 0.05, ** p<0.01 


As expected from the bi-variate cross-tabulation, the (neighbour) national- 
ity variable significantly increases the likelihood of being ‘temporary out’ 
as compared to being ‘always in’ the panel. Controlled for variables related 
to acculturation, however, the positive “temporary out” effect becomes in- 
significant. This implies that the differences shown by nationality observed 
in model 1 are in fact due to the fact that nationals from neighbouring 
countries show higher levels of social participation and interest in politics. 

Unlike for contactability, it appears that more temporary drop-out due to 
non-cooperation by foreigners from neighbouring countries comes from 
those with a medium or longer stay in Switzerland (more than two years). 
Social inclusion variables do not seem to have an independent effect on 
the likelihood of cooperating. 

To summarise, from the cross-tabulations, we mainly find more tempo- 
rary drop-out due to individual non-cooperation patterns among foreigners 
from a neighbouring country and more permanent drop-out due to house- 
hold non-contact among the other foreigners in the SHP. The multivariate 
analyses show that both phenomena are, in part, determined by the dura- 
tion of stay in Switzerland. Foreigners from a neighbouring country with a 
longer stay tend more to temporarily refuse to cooperate. By contrast, those 
who are in the country for a short time tend to drop out permanently due 
to non-contact. Other foreigners tend to drop out permanently due to non- 
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contact, irrespective of duration of stay. Generally, participation of all for- 
eigner groups is worse than participation of native Swiss. 


Representation of foreign minorities in the LFS 


In this section we analyse the relationship between attrition, minorities and 

socio-economic status in much more detail than is possible with data from 

the SHP using the rotating panel component of the LFS. Specifically we 

test if, when controlled for educational level and social class, possible attri- 

tion of national minorities becomes weaker (Schmeets & Michiels 2003; 

Feskens et al. 2007). We use nested binomial logistic regression models.!° 

The following independent variables are controlled for in the models, 

though the coefficients of these are not listed in the tables with the results 

(Tables 11.5 and 11.6): 

— age-group (15-24; 25-39; 40-54; 55-64; more than 65) to capture non- 
linear relationships (Pisati 2003) 

— gender 

— household size, assuming that larger households are less likely to move 
and easier to contact 


We also added control variables related to the survey design, such as the 
number of contact attempts and the wave of the interview, with the under- 
lying hypothesis that the probability of attrition decreases as individuals 
participate in further waves of the survey. 

Model 3 includes the already mentioned control variables and nationality 
recorder in ten categories: 1) Swiss; 2) Italian; 3) Spanish; 4) French; 5) 
Portuguese; 6) German; 7) Turkey; 8) former Yugoslavian and Albanian, 
9) Rest of Europe; 10) Other non-European. Table 11.4 shows the 


Table 11.4 Rate of stay by national minority status in the Swiss LFS sample 


Rate of stay (%) Number of observations 
French 85.37 123 
Swiss 82.60 14,526 
Italian 80.65 672 
Spanish 80.15 131 
German 79.67 241 
Portuguese 78.29 152 
Rest of Europe 77.78 207 
Former Yugoslavia and Albania 75.73 206 
Other non-European countries 72.65 117 
Turkey 71.74 46 
Total 82.19 16,421 


Source: Swiss LFS 2001-2002 
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probability of stay of each of these groups between 2001 and 2002. We 
find that foreign nationalities have a higher likelihood of attrition, except 
for the French. It is especially high among former Yugoslavians and 
Albanians (24 per cent), Turks (28 per cent) and nationals from other non- 
European countries. Nationals from neighbouring countries show similar 
rates of attrition to that of the Swiss. 

In addition to the variables used in model 3, models 4 and 5 include the 
variables that we have hypothesised as mediating the probability of attri- 
tion shown by different foreign minorities. These are education levels (1 
Primary/Secondary I; 2 Secondary II; 3 Tertiary; 4 In training) and occupa- 
tion measured by the International Standard Classification of Occupations 
(ISCO), in four groups as defined earlier in this chapter. Table 11.5 
presents the results from the multivariate models successively controlled 
for education and social class. We expect that attrition decreases with in- 
creasing educational level and social class. 


Table 11.5 Logit model results for ‘drop out’ versus ‘stay’ 


Model 3 Model 4 Model 5 
Foreign groups (reference category: Swiss) 
Italian 0.155 0.116 0.118 
Spanish 0.249 0.197 0.200 
French -0.072 -0.071 -0.065 
Portuguese 0.504* 0.401 0.407 
German 0.139 0.155 0.139 
Turkish 0.799* 0.735* 0.733* 
Former Yugoslavian and Albanian 0.520** 0.486** 0.494** 
Rest of Europe 0.297 0.319 0.321 
Other country 0.701** 0.680** 0.697** 
Education (reference category: Secondary l) 
Secondary II -0.129* -0.118* 
Tertiary -0.195** -0.181* 
In education -0.038 -0.069 


Social class (reference category: Higher non-manual) 


Lower non-manual -0.016 
Manual 0.026 
Not active or unemployed 0.053 
Constant -0.796*** -0.812*** 
N 16,421 16,373 16,308 
Pseudo R2 0.034 0.035 0.035 


Source: Swiss LFS (panel samples 2001-2002) 
Note: Model controlled for age, gender, number of contact attempts, wave and household 
size. Significance levels: * p<0.05, ** p<0.01 
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The explained variance, expressed by the Cragg and Uhler R squared, is 
quite low, ranging from 3.4 per cent to 3.5 per cent with control variables 
included. We find that foreigners, mostly nationals from non-neighbouring 
countries, exhibit higher attrition. Former Yugoslavians and Albanians have 
a 68 per cent higher risk of attrition than Swiss citizens and foreigners 
from “other countries” a 99 per cent higher risk of attrition.'* We find that 
the processes leading to an under-representation of foreigners seem inde- 
pendent of the research variables. In fact, while in the baseline model the 
risk of attrition of former Yugoslavians and Albanians is 68 per cent higher 
than that of the Swiss, when controlling for educational level and social 
class, this figure decreases to 63 per cent, only 5 percentage points fewer. 
The conclusion is that the higher attrition levels of foreigners cannot be at- 
tributed to social class and education. 


Table 11.6 Logit model results for ‘drop out’ versus ‘stay’ 


Education Social class 

National groups National groups 

Swiss (reference category) 0.000 Swiss (reference category) 0.000 

Neighbouring country -0.024 Neighbouring country 0.235 

Other country 0.660*** Other country 0.354 

Education Social Class 

Secondary | 0.000 Higher non-manual 0.000 

(reference category) (reference category) 

Secondary II -0.099 Lower non-manual -0.001 

Tertiary -0.202* Manual -0.034 
Not Active (NA) or 0.095 
Unemployed 


Interaction effects: 


Education*Nationality Social class*Nationality 
Upper secondary* 0.182 Lower non-manual* -0.195 
Neighbouring Neighbouring 
Tertiary* Neighbouring 0.376 Manual*Neighbouring 0.052 
Upper secondary* -0.503* NA or Unemployed* -0.338 
Other country Neighbouring 
Tertiary*Other country -0.039 Lower non-manual* 0.070 
Other country 
Manual*Other country 0.431 
NAor Unemployed* -0.199 


Other country 


N 16,308 N 16,308 
Pseudo R2 0.036 Pseudo R2 0.035 


Source: Swiss LFS (panel samples 2001-2002) 
Note: Model controlled for age, gender, number of contacts attempts, wave, household 
size, type of contract, industry, occupation. Significance levels: * p<0.05, ** p<0.01 
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The previous results showed that foreign minorities have different levels of 
under-representation and attrition and that this under-representation cannot 
be explained by education and social class. However, it is possible that 
within different nationalities attrition varies by education and social class. 
To test this we estimated two models with interaction effects between so- 
cial class and nationality and the other between education and nationality. 
The model has been specified as before, except for a further aggregation of 
nationality in three categories: 1) Swiss; 2) foreigners from neighbouring 
countries; 3) foreigners from non-neighbouring countries. 

Considering education effects, the Swiss or those from neighbouring 
countries with elementary or lower secondary educational levels exhibit a 
similar level of attrition, while foreigners from non-neighbouring coun- 
tries with an elementary or lower secondary degree have a significantly 
higher attrition. For foreigners from non-neighbouring countries, an upper 
secondary degree makes the difference decrease by 39.3 per cent, which 
indicates a moderating effect of education. The interaction effect of na- 
tionality and tertiary education is not significant, due to the small sample 
size. The model therefore shows that the most under-represented are those 
from non-neighbouring countries and especially the lower-educated 
among this group. As for the social class dimension of social exclusion, 
the model confirms the bias shown in the bi-variate analysis above: the 
absence of significant interaction effects between social class and 
nationality. 


11.6 Conclusions and implications 


The present chapter investigated the under-representation of foreign minor- 
ities in social surveys in Switzerland. We distinguished between non-re- 
sponse in cross-sectional surveys and attrition in panel surveys. We fo- 
cused on two aspects related to the differences of under-representation of 
foreigners: the heterogeneity of the countries of origin and the inter-relation 
of education and social class within nationalities. We expected these varia- 
bles to have moderating effects and thus to diminish the under-representa- 
tion of foreigner groups. 

From the results, two main conclusions can be drawn. First, the hetero- 
geneity of national minority populations cannot be adequately analysed us- 
ing one foreigner category. Foreign minorities exhibit different degrees of 
non-response and attrition and each of them probably have different rea- 
sons. Second, we find a large under-representation of foreigners who are 
more subject to exclusion, like former Yugoslavs, Albanians and those 
from non-neighbouring countries. The analyses indicate that while social 
class has no effect on under-representation of foreigners, education seems 
to have a larger impact. Foreigners with the lowest educational levels are 
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more severely under-represented. These results apply both for longitudinal 
(attrition) and cross-sectional surveys. 

Both the cross-sectional and the attrition analyses show that foreigners 
(especially those from non-neighbouring countries) tend to be represented 
to a smaller extent than natives, even if we take educational level into ac- 
count. As for the potential moderating effects of education, we find that 
under-representation is higher for the lower-educated foreigners from more 
“distant” countries. The most under-represented are thus not foreigners in 
general, but mostly the lowest-educated among them. 

We wonder about the social mechanisms that create these results. The 
weak effects of social class suggest that under-representation is not induced 
only by “economic factors”. Rather, the importance of education and na- 
tionality indicate that survey inclusion is related to ‘proximity’ to the ‘core’ 
of society. Different explanations can be attempted: insufficient language 
competences are undoubtedly one barrier to participation (Ngo-Metzger et 
al. 2004). In the ESS, language problems are mentioned by about 3 per 
cent of the total sample, that is, including those not responding. In addi- 
tion, refusal of the respondent might come from a feeling of incompetence 
or lack of interest because they are ‘outside society” at large. This is related 
not only to language skills but also to general knowledge of the national 
and societal context. Being unable and being uninterested in answering sur- 
vey questions are both important factors in refusal. 

Last but not least, the interviewer may play a role. It would probably 
make sense to look at the interactions between interviewers and respond- 
ents. Does the interviewer encourage interest in the survey, for example, 
by tailoring the survey request to the respondent (Groves and Couper 
1998)? There may be cost-benefit calculations (usually the interviewer is 
paid per completed interview), so interviewers may assess whether sample 
members are ‘worth’ the effort. Less investment is made if the assumed 
probability of participation is low and non-participation would be less dam- 
aging because of an anticipated unpleasant interview. 

Three main consequences result from this situation. First, surveys con- 
centrate on prototypical citizens with boundaries defined by the nation- 
state (the ‘spotlighting effect’). Second, surveys produce a distinction be- 
tween well-represented and poorly represented national minorities (‘con- 
trasting effect’), which blurs the differences between and within national 
minorities. Not taking into account parts of the population particularly ex- 
posed to social exclusion has consequences not only for social indicators 
that are estimated from such surveys but also for subjective attitudinal and 
well-being indicators. 

However, there is hope that this situation will change in the near future. 
The samples for the ESS 2010 and the 2011 Swiss part of the 
International Social Survey Programme (ISSP) have already been drawn 
from a new nationally harmonised individual register. Besides virtually 
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complete coverage of the population, the register includes information on 
the nationality of each sample member, amongst other socio-demographic 
variables. In addition, call data of the ESS 2010 and the Swiss ISSP 2011 
will allow for analyses which distinguish between the two main reasons 
for non-response, non-contact and non-cooperation. As a consequence, we 
will be able to analyse non-contact and non-cooperation behaviour for 
sample members of known nationality separately. With a better knowledge 
of the reasons why foreign minorities are more difficult to contact and 
often are harder to convince to participate, both not contacted and not co- 
operating groups can be treated using a specially-tailored survey design. 

The results presented in this chapter lead to some recommendations for 
survey designers and researchers working with general social survey data. 
The most important finding of the present study is that — apart from excep- 
tions that have not been analysed here (e.g. the LFS, after 2003) — social 
surveys in Switzerland are not able to capture the facets of social and liv- 
ing conditions of the resident population with foreign nationality. As a con- 
sequence, generalisations from these surveys are limited to the population 
majority or to the most integrated minorities that reside in the country. 

Further, the results suggest that as long as minorities are more likely to 
be socially excluded, current surveys underestimate indicators like develop- 
ment and inequality indexes. Therefore, an under-representation of certain 
subgroups of foreign minorities such as the lowest educated has implica- 
tions for indicators that refer to the whole population. 

A third implication of the study is related to the limitations of the di- 
chotomy “Swiss versus foreigners”. This has implications for the strategies 
that are used to overcome bias. On this issue, Laganá, Elcheroth, Penic, 
Kleiner and Fasel (2011) have shown that such practices result in an even 
more increased bias as long as they consider the minority population as 
one category. For example, weighting strategies based on the dichotomy 
Swiss versus foreigners results in an over-representation of foreigners from 
neighbouring countries (Italians, French and Germans, who are represented 
as well as native Swiss) while the nationals from “distant” countries still re- 
main under-represented. As a consequence, it is necessary to take the real 
heterogeneity of the samples into account and use specific measures that 
accommodate the heterogeneous groups — both between and within foreign 
minorities. Two promising avenues are oversampling in general surveys 
and designing surveys especially for minorities. 
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Notes 


1  www.bfs.admin.ch/bfs/portal/en/index/themen/01/07/blank/key/01/01.html. Accessed 26 
August 2011. 

2 In this minority we include foreigners from Serbia and Montenegro (184,428), Croatia 
(36,281), Bosnia and Herzegovina (37,631) and Macedonia (59,909). 

3 See chapters 7 and 8 for a longer discussion about non-response in immigrant surveys 
and chapter 9 for the same issue in general population surveys. 

4 The appendix to this chapter provides more details on the similarities and differences 
among the three surveys. 

5 An odds ratio of 0.6 indicates that 40 per cent of the Portuguese individuals with low ed- 
ucation levels are not represented in the LFS. 

6 Acculturation issues like the number of years in the country — although collected, for ex- 
ample in the ESS — are not analysed in the cross-sectional surveys because they are not 
included in the census data. 

7 The reason for this decision is that Voorpostel and Lipps (2011) analysed response behav- 
iour depending on previous change. A change of a variable can only be measured with at 
least two measurements. As we are not interested here in previous change but only in 
characteristics, people who only reported in one wave are also kept in the analyses. 

8 This can in part be explained by their presumably higher probability of moving out of the 
country and thus leaving the sample. If these households cannot be tracked, they are not 
treated as ineligible, but count as drop outs. 

9 The reason for including this variable is that we want to consider the extent to which the 
household is easy to contact as a control variable to analyse cooperation (on the individu- 
al level). 

10 Unlike in linear regressions, if the outcome is non-linear, the coefficients from two or 
more models are not directly comparable. To overcome this limitation we used the meth- 
od recently developed by Karlson, Holm and Breen (2011) (KHB). We used a continuous 
measure of socio-economic status and education. The corresponding coefficients are very 
similar to those presented in Table 11.5. However, since the KHB procedure does not ac- 
cept factors as mediator variables we would have to drop from our sample all individuals 
still in education or out of the labour force, thus estimating the models on different sam- 
ples. For this reason, we use nested binomial logistic regression here. 

11 The odds ratios can be obtained by using the formula: [exp(B)-1]*100. This can be seen 
as follows: If B=logit(p)=log(p/(1-p)) then p=1/(1+exp(-B)) and p/(1-p)=exp(B). Now if 
B=0.520 (see table above) then exp(P)=1.68= p/(1-p) and [exp(f)-1]*100=68 [%]. 
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CONCLUSIONS 


12 Surveying immigrant populations: 
Methodological strategies, good practices 
and open questions 


Mónica Méndez and Joan Font! 


12.1 Introduction 


Our review of survey experiences that include immigrant populations has 
offered important insights into the methodological challenges involved in 
these surveys and the possible research strategies that can be developed to 
deal with them. There are significant lessons that can be drawn from this 
diverse set of experiences with surveys from seven countries, each of 
which had different objectives and resources. To discuss them, we divide 
our conclusions into the same two main areas mentioned in the introduc- 
tion. First, we will deal with issues related to the definition of the target 
group and to sampling design and, secondly, with fieldwork issues. 

Within both of these sections we will look at the reality of surveys of 
immigrants as well as surveys of the general population that include immi- 
grants. Several specific aspects will be analysed. First, we will compare the 
methodological strategies followed by the different surveys covered in the 
book (providing occasional comparisons with cases not included here). 
Second, we will discuss the extent to which the differences that appear 
among these surveys are due to contextual differences (differing immigra- 
tion realities, differences in resources available, etc.). We will see if there 
is a clear-cut best set of practices and recommendations that can be fol- 
lowed by any researcher preparing an immigrant survey or, alternatively, 
whether things are more complex and all we can do is identify the different 
possible avenues that can be chosen depending on the specific circumstan- 
ces of each particular survey. In order to do so, we will mostly concentrate 
on the ideas from this book, but we will also establish a dialogue with the 
ideas and strategies put forward by other researchers. 
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12.2 Deciding on the target population, sampling frames and 
sampling strategies 


Three main aspects of the design of any survey will be discussed in this sec- 
tion: the definition of the target population, the selection of groups to be in- 
terviewed and finally, the sampling frames and sampling strategies used in 
order to build representative samples of the populations to be studied. 


The target population 


The definition of who constitutes the target population of a survey depends 
very much on the aims of the survey, but also on a basic contextual factor, 
namely, the existing data; that is, the administrative records of the immi- 
grant population and how their status is registered. As Jacobs et al. point 
out (2009: 69), “anyone wanting to perform comparative research on immi- 
grants or [immigrant origin] ethnic minorities in Europe is unavoidably 
confronted with the most diverse types of national statistical data and has 
to opt for ad hoc solutions”. 

The surveys analysed in this book have tried to answer questions regard- 
ing issues primarily related to migration and ethnicity. The difference be- 
tween the two concepts has proven to have consequences for the way the 
target population of a survey is defined: migration can be measured 
through ‘objective’ data kept in registers, while the definition of ethnicity 
usually includes certain ‘subjective’ aspects such as feelings of identity 
and of belonging to a group, culture or tradition. 

Most of the surveys presented in the different chapters define their target 
population as the resident population in a given country, thus including im- 
migrants, or they focus specifically on the immigrant population (or on 
some immigrant groups in particular). Generally speaking, the lower age 
limit is set around 15-16 years of age and the upper age limit around 50 
years of age, when there is one, usually mirroring the age structure of the 
population under investigation. In most cases, foreign-born residents are 
the target population. This is the case in two of the three immigrant sur- 
veys carried out in Spain (the Spanish National Immigrant Survey, ENI; 
and the survey done in Madrid within the LOCALMULTIDEM project 
covered in chapters 2 and 7, respectively) and in the surveys carried out in 
the Netherlands, Sweden and Denmark (chapters 5, 6 and 8), though with 
some variations depending on whether they also include descendants of 
foreign-born people. For example, in the survey analysed in chapter 5, the 
target population is the descendants of Surinamese, that is to say, individu- 
als between 15 and 35 years of age with at least one parent born in 
Surinam. By contrast, in the Danish survey discussed in chapter 8, the tar- 
get population is composed of the foreign-born residing in Denmark (from 
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Iran, Pakistan or Turkey) as well as a Danish-born control group, therefore 
excluding the descendants of foreign-born residents. 

The only country covered in the book where the target population of sur- 
veys is usually defined in terms of (self) defined ethnicity is the United 
Kingdom. As Jacobs et al. (2009) point out, the United Kingdom is the 
most notable example of the tradition of studying ethnicity. This might 
have to do with the fact that it is a country of longstanding migration and 
therefore the potential differentiating elements of these groups would not 
be captured by nationality (many of these groups come from former colo- 
nies or have UK nationality anyway) or by place of birth (many of the tar- 
get groups were born in the country and have been in the United Kingdom 
for some generations already). As a result and following the recommenda- 
tion of the Office of National Statistics, in most of the surveys discussed in 
chapter 2, the criterion to define the target population is ethnic self- 
identification. 

What are the potential implications of using only “subjective” indicators 
such as ethnic self-identification in the definition of the target population 
of a survey? If the outcome being examined is related to the subjective def- 
inition of the population under investigation, using subjective indicators to 
define the target population of a survey (and of the research questions it 
tries to provide information to answer) might potentially diminish the var- 
iation observed on the dependent variable (e.g. electoral behaviour, social 
mobility, having experienced discrimination). That may happen, for exam- 
ple, if the aim of a survey is to measure discrimination and it is suspected 
that discrimination is associated with being part of an ethnic minority or 
that feelings of belonging to an ethnic minority increase if one has experi- 
enced discrimination. In that case, it is likely that delineating the target 
population as a subjectively defined ethnic minority might lead to a detec- 
tion of higher levels of feelings of discrimination than would be the case if 
the target population was defined as foreign-born (or as descendants of for- 
eign-born people). 

One way to avoid that potential problem is to collect information on oth- 
er indicators such as country of birth and nationality in the same survey. 
Having both objective indicators such as these and subjective ones such as 
ethnic identification permits researchers to verify whether what matters is 
primarily the subjective feeling of belonging to a minority or the fact that 
one has been born in a different country or has a different nationality in 
the topic being analysed (e.g. voting behaviour, performance at school). 
This debate on whether to use subjective or objective indicators is present 
in other fields of social research, such as social stratification: one can con- 
centrate on measuring class stratification on the basis of occupation or us- 
ing indicators of class consciousness. 

An example that might be useful to illustrate this point is provided in 
chapter 5, which analyses the extent to which the ethnic background of the 
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interviewer matters in the responses given to different survey questions. 
The target population of the survey is composed of individuals resident in 
Amsterdam who have at least one Surinamese-born parent. In this case, 
choosing ethnicity as the basis for the selection of the target population of 
the survey (as would be the practice in the United Kingdom) would not 
have been the best option. Matching interviewers with interviewees who 
subjectively define themselves as belonging to an ethnic group might have 
reduced the variation of reactions to the ethnicity of interviewers, particu- 
larly 1f reactions depend on how intensely they identify with their ethnic 
group. 

All of this does not mean that ethnicity is not an interesting theme to ana- 
lyse. Even if subjective ethnic identification does not play a major role as a 
guiding criterion for the definition of the target population or the sampling 
design in many of the countries covered, it can still be a main thread of 
analysis 1f questions such as the one included in the UK census, asking re- 
spondents to define the ethnic group they belong to, are included in the 
questionnaires. This is the practice followed in the survey carried out in 
Stockholm: though the target population was defined in terms of country of 
origin (including descendants of the foreign-born), the questionnaire in- 
cluded a question asking interviewees about their ethnic identity (see chap- 
ter 6). 

In any case, it is difficult to provide guidelines in this respect, given that 
the decision very much depends on the research interests in each instance. 
It is also interesting to note that reality sometimes defies attempts to distin- 
guish the objective and subjective alternatives. In the Spanish ENI the ‘ob- 
jective” criterion of migrant status was preferred as the variable defining 
the target population. However, some groups that could objectively be de- 
fined as immigrants on the basis of the strict definition (someone who 
comes from a different country and resides or has the intention to reside in 
another country), such as individuals holding Spanish nationality who were 
born in a country other than Spain or individuals from developed countries, 
were less willing to participate in the survey because they (subjectively) 
did not feel that they were part of the target group, not seeing themselves 
as “immigrants” (see chapter 2 in this volume). 


How do we choose the groups to survey? 


Deciding what groups to survey obviously depends on the aims of the sur- 
vey as well as on other practical factors, such as time and material and hu- 
man resources available. In most of the surveys discussed in the chapters 
of this book (and the research projects to which they were associated) this 
decision was greatly conditioned by the financial resources available. 

The Spanish ENI is one on the most comprehensive surveys among the 
ones examined in this book. It was carried out in 2007 and its main goal 
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was to get a panoramic perspective on the immigrant population living in 
Spain. As a result, all immigrant groups were considered, including indi- 
viduals holding Spanish nationality who were born abroad (even if both 
parents had Spanish nationality), provided that they had moved to live in 
Spain after the age of two. By contrast, the NEPIA survey in the Spanish 
region of Andalusia (chapter 4) limited its scope, as its focus was not the 
immigrant population as a whole but only the groups coming from less-de- 
veloped countries. 

The General Social Survey in the United States, the general (resident) 
population surveys carried out by the Centro de Investigaciones 
Sociológicas (CIS, Sociological Research Centre) in Spain, as well as the 
three Swiss surveys examined in chapter 11, also tried to cover the immi- 
grant and foreign population without selecting specific groups. However, 
the main objective of these surveys was to analyse trends among the gener- 
al population residing in these countries, rather than doing specific analysis 
of immigrants and foreigners. 

The option of covering all immigrant groups is not the general rule in 
the immigrant surveys covered by this book; rather, the tendency is to se- 
lect a limited number of groups as target population. In the Danish case 
(chapter 8), the survey addressed immigrants from Pakistan, Iran and 
Turkey, as well as native Danes. Although it is a general population survey, 
the logic of selecting groups is also present in the UK’s Understanding 
Society survey (chapter 2), which incorporated a boost sample designed to 
obtain representative samples of the most numerous non-white ethnic 
groups. 

The selection of the groups to be surveyed is usually based on several 
factors. Some are research-driven, while others also respond to practical 
considerations regarding the context in which surveys are designed and ad- 
ministered. In the case of the survey carried out in Stockholm (chapter 6) 
within the framework of the research project LOCALMULTIDEM, 
Chilean and Turkish immigrants were selected as the target groups because 
Chile and Turkey have been important source countries of migration to 
Sweden. This was also a relevant consideration in the survey carried out in 
Madrid (chapter 7) under the umbrella of the same research project, whose 
target groups are Moroccans, Ecuadorians and a mixed group of people 
born in the rest of the Andean countries. Along with the population survey, 
the project LOCALMULTIDEM also includes a survey carried out among 
immigrants’ associations and organisations. It was therefore important to 
select immigrant groups that had had the opportunity to create an associa- 
tional life of some relevance. 

In most of the other cases covered in this volume, resources were lim- 
ited. Choosing to limit the number of groups surveyed enhanced the proba- 
bility of obtaining quality data (as well as permitting strategies such as 
translation of questionnaires and use of ethnic matching in the selection of 
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interviewers). The numerical relevance of the immigrant groups chosen 
was important both in terms of the substantive interest of findings, but also 
because of practical factors regarding issues such as fieldwork 
organisation. 

An Important consideration stemming from many of the chapters of the 
book (see also Jacobs 2010) is that it is highly recommended to have a 
control (autochthonous) group in the sample, enabling conclusions to be 
drawn regarding the specific effect of ethnicity or the condition of being an 
immigrant on the phenomena analysed. First, from a more descriptive per- 
spective, 1t is useful to have a comparative group of non-immigrants in or- 
der to evaluate the attitudes and behaviours of immigrants. Second, ob- 
served differences in the outcome between immigrants and non-immigrants 
might be explained by other socio-demographic factors that are related to 
both the outcome and ethnic origin.” 

‘Compared to what?” is one of the key questions that any survey analy- 
sis must face. Comparisons can be achieved either by including a control 
group of autochthonous population in surveys primarily addressed to mi- 
grants and ethnic minorities, or by having boost samples of these latter 
populations in surveys that aim to cover the general population. A differ- 
ent, interesting design in terms of comparative analysis is the one that ap- 
pears in ‘both-ways’ surveys, where the comparison is established between 
immigrants in origin and destination countries.* 


Sampling strategies 


Sampling strategies are greatly influenced by the existence of a (good) 
sampling frame. Rather than ‘choosing’ a sampling frame, researchers are 
usually faced with a limited set of alternatives depending on the context in 
which they operate. Some countries have very detailed and accurate admin- 
istrative registers containing information about individuals’ nationality, 
place/country of birth, as well as other data such as parents’ place/country 
of birth. In those contexts researchers have different options for defining 
the target population and sampling methods; they do not have to worry 
about sampling frames. This is the situation in which the surveys in 
Sweden, Denmark and the Netherlands were carried out. As explained in 
chapter 3, in Spain there is a population register that provides a good sam- 
pling frame to start with, especially since improvements in its management 
were introduced in recent years.” However, although it has the advantage 
of including immigrants independently of whether they are residing legally 
in Spain, no information is included about the place of birth of parents. 
The United Kingdom is the clearest example of the other end of the 
spectrum, where there are no good sampling frames to use. As chapter 2 
pointed out, the Electoral Register cannot serve this purpose, particularly 
in regard to immigrants, and therefore anyone carrying out a survey of 
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immigrants and ethnic minorities is confronted with the need to build a 
sampling frame using various strategies. The most promising ones are all 
based on screening areas with a higher percentage of ethnic minority resi- 
dents. But this is a costly process which does not work well with immi- 
grant populations that are geographically dispersed, such as the Chinese in 
the United Kingdom. Other examples of countries which lack appropriate 
sampling frames are France and Italy.° 

In conclusion, it seems clear that contextual aspects regarding the exis- 
tence of good sampling frames matter a great deal.” If there is a reliable 
and accurate register (which does not mean that it will be without any 
problems), it is better to use it and concentrate resources on other stages of 
the survey process. The other advantage of having good registers is that 
they are not only useful for building samples, but also to verify the extent 
to which the profiles of the respondents of a survey resemble the figures 
existing in the register, at least for basic variables such as sex, age and 
country of birth. This is also a good analytical practice (which requires us- 
ing coding categories comparable to the ones in official statistics), which 
we saw used in chapters 4 and 11. 

These contextual differences regarding the existence of sampling frames 
influence the sampling methods that can be used. Designing probability 
samples is the most desired goal, but the lack of a sampling frame, as 
chapter 2 points out, can complicate things a great deal. Given that most of 
the surveys covered in this volume (with the exception of the ones done in 
the United Kingdom and the NEPIA survey in Andalusia) began with a 
good sampling frame, the main decision regarding sampling had to do with 
two aspects: stratification and whether to build a nominal sample or choose 
a different probability sampling method. Regarding the first issue, usually 
some kind of grouping was done in the surveys that attempted to cover the 
whole immigrant population, as shown in chapter 3. Even in countries 
where immigrants or ethnic minorities constitute a comparatively high per- 
centage of the total population, they are still minority groups, and therefore 
usually some kind of grouping of a particular immigrant group, based on 
geographical origin or some other criterion, has to be done to ensure a 
minimum but feasible representation of diversity. For example, in the 
Spanish ENI the stratification meant treating Ecuadorians, Moroccans and 
Romanians as separate groups, while the immigrants of other countries of 
origin had to be grouped together in geographical areas, as it would have 
been difficult to achieve a broad enough sample because of the small num- 
ber of foreign born from other countries, if taken separately.* 

Simple probability samples (with no stratification) were used, for exam- 
ple, in the local surveys addressed to only two or three immigrant groups 
(such as the survey carried out in Madrid and the one in Stockholm, cov- 
ered in chapters 7 and 6, respectively). Even in these cases, difficulties ex- 
perienced in obtaining good response rates forced the survey organisers to 
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change the sampling strategies once the fieldwork had started. These diffi- 
culties illustrate a central idea: even with a good sampling frame as a start- 
ing point and having chosen a simple sampling strategy, fieldwork may 
still be quite difficult, especially if resources are limited. In fact, this is an 
important idea which is often insufficiently highlighted in the survey litera- 
ture: the resources available to start with (money, but also trained inter- 
viewers, time and sampling frames) decisively shape the strategies that can 
be executed and this should be given consideration when choosing the re- 
search strategy in general, and the sampling strategy in particular.” 

As for the choice between a nominal sample and a different sampling 
strategy, the conclusions drawn from this book are less obvious. The main 
advantage of name samples is that it is easier to know what is going on in 
terms of response rate and the ‘fate’ of every single individual included in 
the initial sample. The main disadvantage is that location problems may 
exist and having a name sample with no possibility of substitution may 
lead to a high non-response rate, mainly due to inability to locate individu- 
als (either because they have moved or because their registration was 
‘false’ and they never lived at the address listed in the register). 

This dilemma was faced by the ENI in Spain. Having tried the nominal 
sample in the pilot test, they ended up opting for a sample of dwellings 
where at least one individual born in a foreign country was registered. The 
reason for this is that the results of the pilot test showed that, even though 
immigrant groups were highly mobile, the tendency was for houses and 
residences to be subsequently occupied by other immigrants (in many 
cases of the same geographical origin). Other surveys covered in this book 
that used name samples encountered similar problems in locating immi- 
grant groups (e.g. the Danish survey in chapter 8). 

In the context of a general population survey, the conclusion is also clear 
that a name sample (designed for the administration of the 2008 ISSP survey 
in Spain) meant a much lower response rate among non-nationals, mainly 
due to problems of locating this population (chapter 9). However, chapter 7 
on the survey done in Madrid seems to reach a different result, which shows 
that if sufficient effort is made, the problem of locating immigrants can be 
greatly diminished. Whether it is worth investing a lot of resources in such a 
task or whether it is better to opt for a different sampling strategy from the 
outset, as was decided in the ENI, remains an open question. 

A final consideration is worth noting regarding comparative surveys. 
The introduction discussed the efforts being made by statistical offices and 
international organisations to produce comparable data and the difficulties 
they face. The challenges faced by a small group of researchers trying to 
produce new (comparable) survey data about immigration are of a similar 
nature, but larger and simpler at the same time. They are similar because 
they face the same problems of making comparable what is different from 
the onset (e.g. the quality of the electoral registers or the composition of 
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the immigrant population for the Spanish and Danish teams of the 
LOCALMULTIDEM project). The problems are larger because research 
teams will often be less well-funded and quite heterogeneous in terms of 
expertise and available funding, a difference that becomes even more im- 
portant when projects require the cooperation of research groups from the 
North and the South. However, the problems may also be simpler to solve 
because decisions are in the hands of a small group of people who are usu- 
ally not conditioned by bureaucratic procedures and old traditions, so they 
can adopt comparable strategies from the early planning stages, as shown 
by LOCALMULTIDEM (dealing with immigrant surveys), the European 
Social Survey (dealing with general population surveys) and other compa- 
rative surveys. 


123 How should the interview be conducted? Language, 
fieldwork and response rates 


This section will deal with three basic aspects. First, we will review strat- 
egies, recommendations and open questions about the questionnaire prepa- 
ration stage and its adaptation to multi-ethnic societies. Second, we will 
see how to deal with fieldwork, including aspects such as the selection and 
training of interviewers. Finally, we will address the outcomes of fieldwork 
and the challenge of obtaining good response rates among a highly mobile 
population. 


Language and questionnaires 


Multilingual societies require multilingual surveys. The composition of im- 
migrant communities is in constant flow: at the same time that second gen- 
eration or long-settled immigrants become fluent in the local language(s), 
other immigrants with limited knowledge of this same language will be ar- 
riving. Hospital waiting rooms and train stations have already become mul- 
tilingual and the survey world lags only slightly behind them. The surveys 
included in this book have been sensitive to this multilingual reality and 
used different strategies to deal with it. The most obvious strategy is to 
translate the questionnaire into the languages which are most prevalent 
among immigrants and especially among those groups with greater diffi- 
culties with the local language. ° 

Obviously, the need to adopt such a strategy depends on the number of 
people among the target population that are not fluent in the native lan- 
guage. As a result, in surveys addressed basically to immigrants the use of 
some translation will normally be a must: the LOCALMULTIDEM survey 
in Madrid (chapter 7) used Arabic as a second language; NEPIA (chapter 
4) and ENI (chapter 3) used Arabic, English, Russian and French. The 
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Danish survey in chapter 8 was also available in Farsi, Urdu, Turkish and 
the Swedish one covered in chapter 6 was translated into the languages 
spoken by two of the targeted groups, Turkish and Spanish (for the 
Chilean group). 

The decision becomes more complicated in surveys addressed to the 
general population. If the goal of a survey is to represent a given immi- 
grant group, the crucial question is ‘how many individuals to be inter- 
viewed will not be able to cope with a survey in the local language?” The 
answer to this question will almost automatically determine whether there 
is a need for translation: the survey will be of little use if an important part 
of the community to be represented is unable to understand the questions 
or refuses to participate because of language problems.'' 

In surveys of the general population the crucial question becomes, ‘how 
much will the picture of the general population be distorted if we do not 
translate the questionnaire?” The goal is no longer the small picture, but 
the bias that languages may introduce in the big picture. The answers to 
this question have been very diverse in the surveys analysed in this book, 
from the introduction of Spanish as a second interviewing language in the 
US General Social Survey, to the possibility of conducting the interview in 
several foreign languages provided in the British survey Understanding 
Society (see chapter 2) and in the three Swiss surveys covered in chapter 
11, to the non-adoption of any second language in the case of the surveys 
carried out by CIS on the general population in Spain (chapter 9). 

Decisions about these different strategies for dealing with language bar- 
riers are related to at least three contextual factors. We have already pre- 
sented the first and most obvious: the larger the size of the potentially ex- 
cluded population (and thus, the larger the potential bias of the data ob- 
tained), the larger the likelihood that translation is absolutely necessary. 
Two other factors will also shape the decision to translate: how important 
are each of the ethnic groups and how likely is it that they will not be flu- 
ent in the language of the country where they are living? In this regard, 
one country that has appeared in several of our chapters (Spain) is more 
the exception than the rule, with most of the crucial foreign-born popula- 
tion groups having Spanish as their own native language (immigrants from 
Latin American countries) or having very few problems in understanding it 
(e.g. Romanians, as shown in chapter 9).'* The result is that the need for 
questionnaire translation is relatively small in the Spanish context, where 
most immigrants can cope with Spanish. In addition, the decision to trans- 
late questionnaires in the Spanish context would raise additional issues. 
Should the survey organisation give priority to the large percentage of the 
Basque population that speaks Spanish but would prefer to be interviewed 
in Basque, or the smaller number of Moroccans which to a great degree 
speak and understand little or no Spanish? Or should priority go to the 
British residents who tend to be less fluent in Spanish? 
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The combination of these contextual factors produces difficult obstacles 
to the use of translation in certain types of surveys, for example, general 
population surveys in countries such as Britain. Britain has a large number 
of non-English-speaking minorities, each representing a very small part of 
the sample. In such situations we cannot expect questionnaire translation to 
become a general policy, and except in extremely large samples or ex- 
tremely well-funded surveys, other less optimal strategies will be used.'? 

Funding'* is precisely the final contextual factor that is likely to influ- 
ence the decision to translate or not, which may also help to explain why 
the ENI questionnaire was translated into more languages (four) than other 
less well-funded surveys, and why the GSS might cope more easily with 
this issue than other surveys addressed to the general population (with 
everything translated into a second language — Spanish). 

Resources are never unlimited. Because they may be better used in other 
aspects of the collection process (e.g. to optimise sampling or response 
rates), a crucial question is whether translation efforts pay off. In some 
cases, the question is meaningless: without translating, the survey of spe- 
cific groups of immigrants would be simply impossible or would produce 
extremely biased data. However, the GSS produced estimates of the US 
population up to 2006 without using Spanish and the surveys done by CIS 
continue to produce data on the Spanish resident population without using 
Arabic. The answer to this question, thus, has to be case-specific and de- 
pends on the number of individuals who speak each language (and are un- 
able to speak the local one). In any case, in cases such as the GSS, the in- 
troduction of Spanish has made a huge difference: 225 people answered in 
Spanish and around a third of them would not have been able to participate 
in the survey if the translation had not been available. More importantly, 
the picture of the Hispanic population in the United States obtained is now 
more accurate, revealing this population group to be substantially different 
and not nearly as assimilated as was previously thought. The quantitative 
impact of the translation of the questionnaire into Spanish is such that 109 
among 140 attitudinal indicators of the overall US population are now dif- 
ferent from before the introduction of this practice (chapter 10). Previous 
research has also emphasised this point (see Bizier, Kaddatz & Laroche 
2004). 

How great the need is for an additional language can only be known 
through previous experience. The ENI survey in Spain is an illustrative 
case. Following the translation of the questionnaire into four languages, 
there was only very limited usage of most of the translations, a situation 
that also arose previously (DaVanzo, Hawes-Dawson, Burciaga & Vernez 
1995). 

Making the choice for a translated questionnaire does not end the story. 
Important decisions need to be made about how to do a reliable translation 
that will produce results that are comparable among the different 
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languages. The GSS again is a good illustration of a cooperative team ef- 
fort involving several stages and reviews adopted to ensure comparabil- 
ity.!* In spite of these efforts, a rigorous test of differences points to unex- 
plained language differences that may be attributable to imperfect compara- 
bility among words or response categories. In short, we make progress by 
translating, but the door we open with it requires extended efforts to guar- 
antee the production of quality data (Harkness, Braun, Edwards, Johnson, 
Lyberg, Mohler, Pennel & Smith 2010). 

An obvious open question is how far we should go to include minority 
languages. On the one hand, there will always be ever smaller minorities 
and groups that would prefer us to use their language ° and there will al- 
ways be individuals who do not understand if we do not use their mother 
tongue, producing a certain degree of bias.” On the other hand, we have 
seen that a reliable and accurate translation is not easy to produce (chapter 
10). The effort involved usually implies that the decision to translate is 
made only when a relatively sizeable proportion of the population is af- 
fected (e.g. the 5 per cent rule used by the European Social Survey where- 
by questionnaires must be translated only into languages spoken at least by 
that percentage of a participant country’s population). 

In conclusion, the primary general recommendation regarding languages 
is to translate questionnaires and other survey documentation (as well as 
the invitation letter and show cards) into as many foreign languages as pos- 
sible (and as reasonable). The chapters in this book have clearly shown 
that this is necessary in many cases and that it leads to a more accurate pic- 
ture of reality even in surveys of the general population. However, the dif- 
ficulties involved in the quality translation of questionnaires and the com- 
parability problems that they raise, even with good translations, is opening 
the door to the exploration of new survey practices, such as using anchor- 
ing vignettes (images that reflect the idea being questioned), that have pro- 
duced quite promising results (Hopkins & King 2010; see also chapter 2 in 
this volume). 

Although language is by far the most important issue in preparing a mul- 
ticultural questionnaire, the surveys analysed in this book also refer to an- 
other essential practice in questionnaire development: doing a pilot study. 
In several of the surveys discussed a pilot study was carried out, resulting 
in relevant changes and consequences for fieldwork preparation. For exam- 
ple, as a result of the pilot test, the researchers in charge of the Swedish 
survey discussed in chapter 6 found that the questionnaire was far too ab- 
stract and long. It was subsequently reduced by 40 per cent. Both the sur- 
vey of Surinamese (chapter 5) as well as the ENI survey (chapter 3) devel- 
oped two pilot studies with different goals and outcomes. In the first case 
they were especially useful for questionnaire development and in the sec- 
ond they were crucial in making the decision to change the sampling unit 
from individuals to households. The lesson is quite straightforward: if pilot 
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testing is advisable in any survey, its importance increases markedly when 
we face new territories, in terms of population, language or culture. 


Interviewer selection and fieldwork strategies 


To make the most of translated questionnaires, the use of bilingual inter- 
viewers is recommended. However, in many cases it will be difficult to 
find trained and professional bilingual interviewers. This produces a dilem- 
ma: is 1t better to work with professional interviewers (and provide them 
with bilingual help when needed) or is it more important to be culturally 
sensitive and use linguistic and ethnic matching, even if it is at the cost of 
using interviewers with less experience? 

The surveys included in this book chose different options and, in most 
cases, research teams in charge of their organisation were satisfied with the 
results. For example, the NEPIA survey in Andalusia and the survey car- 
ried out in Stockholm represent two quite different strategies that obtained 
satisfactory outcomes, according to the conclusions reached in the respec- 
tive chapters of this volume (chapters 4 and 6). The NEPIA case chose 
proximity and trained a new pool of interviewers, trying to adapt ethnic 
composition to the dominant ethnic groups in each of the areas surveyed. 
Since the use of non-professional interviewers involves more risks, a strict 
follow-up strategy was needed, including dismissals of interviewers in a 
few cases. The general results were good, with low levels of non-response 
and a high proportion of phone numbers obtained, even among a popula- 
tion that included unauthorised immigrants. In such a case, the creation of 
a new fieldwork team involves not only intense training, but also difficul- 
ties in recruiting good interviewers and a high risk of staff turnover among 
a group of workers who are likely to have limited experience (see chapter 
n! 

The use of professional interviewers was the option chosen by the other 
Spanish survey (ENI), as well as by the two Scandinavian ones (Sweden 
and Denmark). In these cases, professional interviewers did most of the 
work and only used native interpreters when the individuals being inter- 
viewed had difficulties with the language. The Danish chapter is the one 
that most clearly acknowledges that this was not an ideal solution, as it 
made the fieldwork more complicated and, as a result, extended its length 
because a substantial number of interviewers resigned. Again, the use of 
bilingual help was varied, depending on the composition of the immigrant 
population. There was substantial usage in Denmark, especially in inter- 
views with Arab women, but very limited usage in the case of the ENI. 
Chapter 6, on the survey done in Stockholm, makes a very clear case for 
the benefits of using professional interviewers and suggests that their con- 
tribution can go far beyond interviewing to also making important sugges- 
tions on the organisation of the overall survey process. ? 
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Beyond the complexity of the fieldwork process, does the strategy of 
ethnic matching change any of the results obtained? Our conclusions re- 
garding this subject coincide with earlier literature:?? mixed findings and 
changing results depending on the subject analysed. For example, chapter 
8 claims that interviewer characteristics did not affect results in the case of 
the Danish survey examined. The test of ethnic matching effects is the cen- 
tral contribution of chapter 5: using a carefully tailored design, where half 
of the interviewers were Dutch and half were Surinamese, interviewees 
were assigned to specifically evaluate the effects of ethnic matching. 
Significant differences appeared on most questions, except the primarily 
factual ones. Some of these differences disappeared when appropriate con- 
trols were introduced in the analysis, but the effects of social desirability 
on culturally sensitive questions continued to hold. 

In conclusion, the diverse findings of the chapters in this volume point 
to a limited set of recommendations regarding the characteristics of inter- 
viewers and fieldwork organisation, which are more relevant for surveys 
addressed primarily to ethnic minorities and immigrants. First, the decision 
of whether to work with professional interviewers (with bilingual help) will 
depend on at least two factors: the availability of a professionally trained 
network of interviewers and the complexity of the distribution of immi- 
grant groups and their languages over the field area. Bilingual support will 
be relatively easy to organise when researchers have to deal with two lan- 
guages clearly clustered in given areas. It will be much more complicated 
if they need to cover a large number of languages distributed in a highly 
dispersed population. In any case, to build an alternative pool of well- 
trained interviewers is always costly and one must expect instability and 
difficulties, even when interviewers are well paid. 

Second, given the complexity, the advantages of ethnic matching are far 
from clear. The best solution will depend on the survey topic. For factual 
questions addressed to a group clearly distrustful of other communities, 
ethnic matching may make sense, but its usage in surveys dealing with cul- 
turally sensitive subjects may introduce new sources of bias. As previous 
research has shown (Durrant, Groves, Staesky & Steele 2010), it is not on- 
ly ethnicity but general similarity between interviewers and interviewees 
which becomes crucial for attaining higher cooperation rates. Therefore, 
ethnicity should be taken into consideration together with other socio-dem- 
ographic characteristics of interviewers (e.g. gender and age), especially in 
certain circumstances (e.g. surveys on sensitive topics). 

In any case, quite a number of questions remain open and we will men- 
tion just a few of them. First, we need to know more about the training 
strategies that are most needed and effective, both for professional inter- 
viewers who need to adapt to groups with characteristics different from 
those of the majority, and for new interviewers who belong to the inter- 
viewed community but tend to lack technical skills and experience. 


CONCLUSIONS 285 


Second, the surveys included in this book were mostly conducted face to 
face, and we need to know more about the potential effects of alternative 
modes of administration among immiəgrant groups. Third, we need to learn 
more about the effects of lower-cost strategies such as ad hoc translations 
by interviewers and letting interviewees rely on help provided by relatives. 
Then we will have a clearer idea of whether these strategies should ever be 
recommended or could be used in less well-funded projects or particularly 
complex circumstances. 


Outcomes: Response rates 


After all the efforts described in the previous pages, was it possible to 
achieve survey results based on a high response rate? Again, the surveys 
covered in this book allow for different answers to this question. Response 
rates varied greatly among surveys and, in some cases, among countries of 
origin, allowing for an exploration of the possible factors that lie behind 
these differences. 

The highest response rate appears in the ENI survey, which reached 67 
per cent. This rate decreases substantially in the two Scandinavian surveys 
and even more in the LOCALMULTIDEM project and the 2008 
International Social Survey Program (ISPP) survey in Spain (see chapter 
9), where response rates did not reach 40 per cent, considering only the in- 
dividuals in the sample without Spanish nationality. However, any compar- 
ison must be done considering that the ENI calculation does not follow 
WAPOR common procedures, since the survey incorporated the possibility 
of substitution for some specific circumstances of non-contact. In spite of 
that, there are at least three sets of factors that must be considered to under- 
stand differences in these outcomes. The first one is again resources and 
the fieldwork instructions associated with them. The high ENI response 
rate cannot be understood without considering the importance given by the 
Spanish National Statistics Institute (INE, in Spanish) to that survey and 
the amount of resources devoted to it.?! In addition, since the INE is a 
well-known institution and many of its surveys have response mandated by 
law, it also has a legal and symbolic advantage that most other surveys of 
immigrants lack.” 

A second important explanatory factor is the objective difficulties of the 
populations being targeted. Factors such as the living conditions of the im- 
migrant group being considered,” their average time in the country, their 
degree of mobility and the urbanisation of the area where they live, all con- 
tribute to make the fieldwork remarkably more or less difficult. Some of 
these factors are probably behind the unexplained differences that continue 
to appear among different immigrant groups showing quite different re- 
sponse rates (e.g. from 63 per cent among immigrants of Iranian origin to 
42 per cent among Pakistanis in the same Danish survey, chapter 8). The 


286 MÓNICA MÉNDEZ AND JOAN FONT 


analysis of three Swiss general population surveys presented in chapter 11 
is especially interesting. We see the effect of other sociological variables, 
because it distinguishes the influence of education and occupation from the 
role played by belonging to different nationalities in explaining survey par- 
ticipation. It shows that foreigners from neighbouring countries that share 
one of the Swiss official languages have similar problems and similar rep- 
resentation rates in surveys as the Swiss population, whereas under-repre- 
sentation is higher among the rest of foreign minorities. The larger the cul- 
tural gap (as e.g. in the case of nationals from Yugoslavia’s successor states 
or from Turkey), the larger the under-representation of foreign minorities. 
This gap is only partially reduced through controlling for education. 

Finally, it is crucial to know the reasons why so many interviewees can- 
not be interviewed. Several of the chapters in this volume provide interest- 
ing information in answer to this question. A result that appears in all of 
them and was previously noted is that an important part of non-response is 
due to a failure to establish contact: regardless of the country of origin, res- 
idence or fieldwork protocol, immigrant populations are always difficult to 
reach and quite mobile, and any address record quickly becomes outdated. 

Differences appear in the other reasons for non-response: inability to an- 
swer will be more prevalent when there is no help for non-native language 
speakers (5.6 per cent in the ISSP survey carried out by CIS in Spain, 
chapter 9) than when these resources are available (1.4 per cent in the ENI 
survey, chapter 3). However, the most important difference appears in co- 
operation rates, ranging from the extremely cooperative immigrant group 
in the ISSP sample (3.5 per cent of non-cooperation) to a maximum of 
around 30 per cent in the Swedish survey covered in chapter 6 of this 
volume. 

In those cases where cooperation rates can be compared with the native 
population, contradictory findings continue to appear, with the Spanish re- 
sults more in line with previous evidence that found greater cooperation 
from immigrant populations than shown in the results of the Scandinavian 
surveys in this book,** where cooperation rates are slightly higher for the 
native populations. The analysis of the presence of immigrants and foreign 
groups among respondents of several Spanish general population surveys 
in chapter 9 reveals interesting evidence that suggests that the lower incli- 
nation to cooperate shown by the native, compared to foreign-born popula- 
tion, may be partially due to the fact that the former have been exposed to 
too many surveys. 

Good practices that help to achieve higher response rates are not radi- 
cally different from those used in any other survey: increasing the number 
of contact attempts (up to a certain point) and spreading these attempts 
over different days and times, establishing incentives for interviewers and 
interviewees, sending advance letters and using well-trained interviewers. 
The differential aspect that immigrant populations introduce (higher 
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mobility and as a result, higher non-contact rate) can be dealt with either 
through a change in the sampling unit to households (under the assumption 
that often the new people living in a household will continue to be immi- 
grants, as the ENI did) or by considering response rates shown by these 
groups in previous surveys when doing the sampling design (the response 
rate would remain the same, but researchers would achieve their target 
number of interviews through differential oversampling). The different be- 
havioural patterns of each of the immigrant groups means that, as chapter 
7 argues, surveys aiming to cover native and immigrant populations should 
probably choose different sampling and fieldwork strategies for each of 
them in order to obtain similar outcomes. The experience of the ESS over 
the last decade shows that maximum comparability cannot be achieved by 
always using identical procedures, but by adapting procedures to the needs 
and possibilities of each particular setting and with a rich and detailed 
process of documenting every aspect and decision of all the survey process 
steps. 

Some of the data presented has raised more questions than answers. For 
example, we have collected quite reliable evidence of response rates and 
reasons for non-response for different immigrant groups in different cities 
and using different fieldwork strategies. However, given the considerable 
discrepancies in the response rates, we have been able to understand only a 
small part of where they come from. The factors Just discussed are clearly 
important, but unanswered questions remain. Why, for example, in 
Sweden, do men of all ethnic groups have lower response rates than wom- 
en, while this pattern only emerges among certain ethnic groups in 
Denmark? Only with additional research will we be able to answer such 
questions. 


12.4 Looking ahead 


How should we proceed to achieve quality survey data on immigrants? 
The different surveys examined in this book have made it clear that it is 
impossible to provide universal recipes that work in all circumstances. In 
spite of that, we have tried to provide two kinds of recommendations. 
First, a list of good practices that have worked well in the surveys exam- 
ined throughout this volume, the feasibility and external validity of which 
are discussed in this concluding chapter. Most of them cannot be applied 
in all circumstances, but we have shown their advantages and limitations 
and discussed the situations in which they may be appropriate. Second, the 
topics that we have dealt with in this conclusion can be used as a kind of 
checklist of issues to be considered when planning a survey involving im- 
migrant populations. It may not be easy to provide answers on what to do 
in all circumstances, but at least we have tried to provide a thorough list of 
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relevant questions that have to be kept in mind when planning a survey ad- 
dressed to immigrants or ethnic minorities. 

To be sure, more methodological research is needed on this topic. The 
demand for more data about immigrants and ethnic minorities from nation- 
al and supra-national bodies makes us confident that the number of surveys 
addressed to these populations will increase in the future. Along with them 
new insights will emerge on how to address the methodological challenges 
that these types of surveys involve. 

Finally, it is important to widen the scope and to consider that surveys 
are an important instrument, but just one of the tools available to obtain 
data about immigrants and ethnic minorities. If the general goal is to know 
more about the reality of these populations, as Jacobs (2010) advocates, 
survey data should be combined with a more thorough usage of data regis- 
tered in administrative databases, provided that privacy protection is guar- 
anteed, as well as with other types of data collection such as qualitative 
studies. 


Notes 


1 The views expressed in this chapter are personal and do not necessarily reflect those of 
the organisations to which the authors belong. 
Encuesta Nacional de Inmigrantes. 

3 This good practice is respected by most of the surveys in this book, except for those with 
mostly descriptive aims, such as the NEPIA and ENI surveys, whose goals were to get a 
first impression of the characteristics of immigrants. 

4 See the chapters that appear in Bonifazi, Okólski, Schoorl and Simon (2008) as well as 
Beauchemin and González Ferrer (2011). 

5 The deficiencies in its updating caused the researchers in charge of NEPIA to build their 
own frame, as explained in chapter 4. That chapter concludes that if the survey had to be 
redone, they would now use the ‘Padrón’ (population register), as it is now kept more up 
to date than it used to be, the only reservation being that immigrants who do not reside le- 
gally in the country are less likely to register for various reasons (lack of knowledge, fear 
of the consequences of being included in a register, etc.). 

6  Blangiardo developed the centre sampling method as a sampling strategy to build samples 
of immigrant minorities and overcome the lack of an adequate sampling frame for that 
purpose in Italy (Blangiardo 1996; Mecatti & Migliorati 2003). For the French case, see 
Tiberj (2006) which examines the strategy followed in the survey “Rapport au politique 
des Français issues de l'immigration”, in order to obtain a sample of French citizens from 
African and Turkish immigration. 

7 Having a good sampling frame does not preclude other types of problems, as immigrant 
populations are usually characterised by a high degree of residential mobility, which may 
not always be captured by the registers. 

8 See chapter 3 for a detailed definition of the groups. 

9 In their assessment of the NIDI project, Groenewold and Bilsborrow (2009) stress the 
idea that research and sampling strategies should only be decided once all funding sources 
have been secured. 

10 The role of bilingual interviewers will be addressed in the next section. 
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Questionnaire translation will be necessary for Bangladeshis in Britain or British citizens 
in Spain, to mention just two of the examples that appear throughout this book, where a 
large part of the immigrant group is not able to speak the local language. 

A similar situation occurs with people from India in the United Kingdom (see chapter 2). 
For example, allowing bilingual interviewers or family members to translate the 
questionnaire. 

The same argument applies for other kinds of crucial resources. For instance, time con- 
straints will limit the adoption of many of the practices we recommend in these 
conclusions. 

Chapter 2 also presents an adaptation of the TRAP model (used by the ESS), which is be- 
coming the new international standard for translation procedures, used in British ethnic 
minority surveys. 

This may be the case for the Kurd community covered in chapter 6: the citizen survey 
was only translated into Turkish that could be understood by all, but the organisational 
survey that had to be completed by more culturally sensitive local elites was also trans- 
lated into Kurdish. 

Lee, Nguyen, Jawad and Kurata (2008) show that the use of Spanish in the GSS is not 
enough in the United States to achieve representativeness since results for Asian people 
(about 4 per cent of the US population) continue to be biased unless additional languages 
are used. 

In fact, the author of chapter 4 expresses doubts as to whether the same strategy would be 
adopted if they were to start again from scratch. 

Previous research repeatedly highlights the crucial role played by experienced inter- 
viewers. See, for example, Durrant, Groves, Staetsky and Steele (2010). 

Feskens, Hox, Lensvelt-Mulders and Schmeets (2006) consider ethnic matching to be an 
inefficient strategy, at least for general population surveys and even problematic for cer- 
tain survey subjects. Chapter 2 in this book also briefly reviews previous findings. 

The different resources between the ENI and the ISSP survey in Spain are evident in 
budgets, numbers of interviewers and other indicators. To give a single indicator, the 
fieldwork lasted almost four months in the case of ENI and slightly more than one month 
in the ISSP survey. 

However, even if a high number of visits of different hours and days are a good practice 
that helps to improve response rates, the LOCALMULTIDEM survey done in Madrid 
shows that even applying the strict European Social Survey rules, response rates may end 
up being quite low. 

For example, the NEPIA survey in Andalusia included immigrants who were not regis- 
tered, whereas the Scandinavian and ENI surveys used official registers to build the sam- 
ple frame. 

In their comparative analysis of survey data of ethnic minorities Feskens et al. (2006: 
300) mention that even though they are also increasing, non-response rates due to refusals 
are usually lower among ethnic minorities than among the native population. 
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